Difference between revisions of "SME Server:Documentation:Developers Manual:Chapter14"

From SME Server
Jump to navigationJump to search
(Splitting the SME Server Developers Guide (and relocating))
 
 
(29 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 
<div class="CHAPTER">
 
<div class="CHAPTER">
=The SME Server development environment=
+
==The SME Server development environment==
  
 
<div class="SECT1">
 
<div class="SECT1">
  
==Configuring your development environment==
+
===Configuring your development environment===
  
The SME Server source code is checked into CVS at SourceForge. The project has a SourceForge page at https://sourceforge.net/projects/smeserver/. All packages are built using '''mezzanine''', and an overview of mezzanine is provided here: https://wiki.caosity.org/tiki-index.php?page=Package+Maintenance
+
The SME Server source code is checked into CVS at koozali.org.
 
+
SME Server code is stored in the CVS on shell.koozali.org in two repositories:
<div class="SECT2">
+
* [https://viewvc.koozali.org/smeserver/ CVS SME Server] which holds the core packages of the SME Server
----
+
* [https://viewvc.koozali.org/smecontribs/ CVS SME Contribs] which holds the contribs packages
 
 
===Reminder: The SME Server is released under the GPL===
 
  
 +
Reminder:
 
The SME Server source code is released under the GPL. You must release the source code to all modifications. If you make improvements, please raise a bug and attach a patch so the change can be discussed and pulled back into the base for everyone to share.
 
The SME Server source code is released under the GPL. You must release the source code to all modifications. If you make improvements, please raise a bug and attach a patch so the change can be discussed and pulled back into the base for everyone to share.
  
</div><div class="SECT2">
+
Only developers who are going to put patches back into CVS and build new packages need shell.koozali.org CVS access. The sources are freely available and patches are gratefully received. Just follow the instructions in this section and attach the patch(es) to the Bugzilla entry, explaining why the change should be made.
----
 
 
 
===Do I need a SourceForge account?===
 
 
 
Not really. Only developers who are going to put patches back into CVS and build new packages need SourceForge CVS access. The sources are freely available and patches are gratefully received. Just follow the instructions in this section and attach the patch(es) to the Bugzilla entry, explaining why the change should be made.
 
  
 
</div><div class="SECT2">
 
</div><div class="SECT2">
 
----
 
----
  
===Add some useful aliases===
+
====Local environment====
 +
* Install cvs
 +
yum install cvs rsh
  
Mezzanine needs to know to use the SourceForge CVS repository in order to checkout/get or import a package. Once a package is checked out from SourceForge, mezzanine/CVS will remember where it came from and check it back into the same CVS repository. You should not set the CVSROOT environment variable as it makes it too easy to accidentally import code into the wrong CVS repository when working with multiple repositories.
+
* Setup CVS to use ssh by creating /etc/profile.d/smebuild.sh with the following content
 
+
Add the following lines to <tt class="FILENAME">~/.bashrc</tt> (or a file such as <tt class="FILENAME">/etc/profile.d/smebuild.sh</tt> to configure it for all users on this machine):
+
# Developer environment
 
+
# This gets symlinked into /etc/profile.d  
  <nowiki>export CVS_RSH=ssh # tell CVS to use ssh
+
 +
  export CVS_RSH=ssh # tell CVS to use ssh
 
   
 
   
 
  # DO NOT set CVSROOT
 
  # DO NOT set CVSROOT
sf_cvsroot=smeserver.cvs.sourceforge.net:/cvsroot/smeserver
 
sf_cvsroot_anon=:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver
 
 
   
 
   
  alias mzgetsf="mzget --dir $sf_cvsroot"
+
  alias rm='rm -i'
  alias mzimportsf="mzimport --dir $sf_cvsroot"
+
  alias cp='cp -i --preserve=timestamps'
+
  alias mv='mv -i'
  alias mzgetsf_anon="mzget --dir $sf_cvsroot_anon"</nowiki>
+
You have to logout and login again to the console for changes to take effects.
 
 
You can now use the '''mzgetsf_anon''' (or '''msgetsf''' if you have a developer account) to retrieve a package from SourceForge. If you use '''mzget''', you will get an error message to remind you that CVSROOT has not been set, and should not be set. Note that you can use the other mezzanine commands, such as '''mzclean''' and '''mzsync''' normally as the CVS repository is already known.
 
 
 
[gordonr@smebuild tmp]$ mzget smeserver-yum
 
cvs checkout: No CVSROOT specified!  Please use the `-d' option
 
cvs [checkout aborted]: or set the CVSROOT environment variable.
 
An unknown error must have occured, because the command returned 256
 
  
</div><div class="SECT2">
+
====Access to build system====
----
+
Check updates/status on the build server: http://buildsys.contribs.org
  
===Create your work area===
+
{{Warning box|msg=For contrib builders using the plague-client-0.5.0 you will need to downgrade to  correct version, to fix this please do:
  
It is a good idea to perform all of your checkouts in a standard directory hierarchy. This makes it easier to move around between the packages. On a shared build server, it also makes it easier for people to recover your work if required (for example when you are on holidays). If you want to use filesharing to edit files from a networked PC, you might like to use <tt class="FILENAME">~/home/smeserver/</tt>. On a shared build server, you might even use something like <tt class="FILENAME">/builds/users/$LOGNAME/smeserver/</tt> so the files are not in your home directory and are accessible to all developers.
+
rpm -e plague-client plague-common
 +
yum --enablerepo=smecontribs install plague-client}}
  
<div class="NOTE"><blockquote class="NOTE">
+
Ask admin@contribs.org for certificates, give the email address to use for notifications.
 +
yum --enablerepo=smecontribs install plague-client
  
'''Note: '''Reminder: All files must be checked in using Unix text format.
+
From now, do not use account "root" anymore. Use a dedicated dev account.
 +
Copy certificates and config file to ~/
 +
.plague-client.cfg
 +
.username.pem
 +
.contribs-upload-ca.pem
 +
.contribs-server-ca.pem
  
</blockquote></div>
+
Don't forget to set the proper privileges on the file
 +
chmod 600 .username.pem
  
  [gordonr@smebuild ~]$ mkdir ~/smeserver
+
Check it's working
 +
  plague-client list_builders
 
   
 
   
  [gordonr@smebuild ~]$ cd ~/smeserver
+
  Builders:
 
+
------------------------------------------------------------------------------------------
</div></div><div class="SECT1">
+
  build64-1.contribs.org      x86_64 amd64 ia32e noarch i386 i486 i586 i686 athlon available
----
+
  build32-1.contribs.org      i386 i486 i586 i686 athlon noarch  available
 
 
==Modifying an SME Server package==
 
 
 
<div class="SECT2">
 
 
 
===Raise a Bugzilla entry===
 
 
 
Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug <span class="emphasis">''before''</span> you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.
 
 
 
<span class="emphasis">''All changes must have an associated Bugzilla entry.''</span> The bug tracker is here: http://www.contribs.org/bugzilla/
 
 
 
If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:
 
 
 
For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" [http://www.contribs.org/bugzilla/show_bug.cgi?id=1174/ http://www.contribs.org/bugzilla/show_bug.cgi?id=1174].
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===Choose the package(s) to modify===
 
 
 
If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run '''rpm -qf''' on the file to be modified:
 
 
 
[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys
 
smeserver-yum-1.1.2-05
 
 
 
and so, we want to modify the <tt class="FILENAME">smeserver-yum</tt> package.
 
 
 
<div class="NOTE"><blockquote class="NOTE">
 
 
 
'''Note: '''You can view a complete list of the packages checked into SourceForge CVS http://smeserver.cvs.sourceforge.net/smeserver/. On rare occasions, the sources of particular packages may be slightly out of date as there is a small delay (typically only a few hours) between changes to the developer CVS and the anonymous CVS. In practice, this rarely matters and at worst it requires a merging of your changes with any other recent changes to the package.
 
 
 
</blockquote></div>
 
 
 
<span class="emphasis">''All packages on the SME Server ISO/CD must be checked into SourceForge CVS''</span>. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===mzgetsf_anon: SourceForge anonymous CVS checkout===
 
 
 
You can now retrieve one of the packages from SourceForge. In this case, we want to modify the <var class="LITERAL">smeserver-yum</var> package, so let's retrieve it from SourceForge:
 
  
[gordonr@smebuild smeserver]$ mzgetsf_anon smeserver-yum
+
====CVS shell.koozali.org access====
U smeserver-yum/ChangeLog
+
* SME Server code is stored in the CVS on http://shell.koozali.org. To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org. With this account the development team can give you access to the CVS repository.
U smeserver-yum/F/smeserver-yum.spec
 
U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch
 
  U smeserver-yum/P/smeserver-yum-1.2.0-DisplayStatus.patch2
 
U smeserver-yum/P/smeserver-yum-1.2.0-ModifyUpdateDBs.patch
 
U smeserver-yum/S/smeserver-yum-1.2.0.tar.gz
 
 
 
<div class="NOTE"><blockquote class="NOTE">
 
 
 
'''Note: '''Use '''mzgetsf''' if you have a SME Server developer account.
 
 
 
</blockquote></div></div><div class="SECT2">
 
----
 
  
===Mezzanine package hierarchy===
+
* After your Koozali.org account has been created you can ask the development team to give you developer access to smecontribs. Create a bug in the Bug Tracker as usual.
  
[gordonr@smebuild smeserver]$ cd smeserver-yum/
+
* If local username is different to shell.koozali.org username edit ~/.ssh/config:
 
   
 
   
  [gordonr@smebuild smeserver-yum]$ ls
+
  Host  shell.koozali.org
  ChangeLog CVS F  P  S  smeserver-yum
+
  User koozaliusername ''(without @shell.koozali.org)''
 
+
  port 222
If you examine the checked out package, you will see the following directories:
 
  
* The SME Server project does not use the <tt class="FILENAME">ChangeLog</tt> file as we maintain all of our changes in the <var class="LITERAL">%changelog</var> section of the <tt class="FILENAME">SPEC</tt> file.
+
* Don't forget to set the proper privileges on the file
* The <tt class="FILENAME">CVS</tt> directory contains CVS state information, such as the location of the CVS repository: [gordonr@smebuild smeserver-yum]$ ls CVS
 
Entries  Entries.Log  Repository  Root
 
* The <tt class="FILENAME">F</tt> directory contains the <tt class="FILENAME">SPEC</tt> for this package: [gordonr@smebuild smeserver-yum]$ ls F
 
CVS  smeserver-yum.spec
 
 
* The <tt class="FILENAME">P</tt> directory contains any patches which are required to update the source in the tarball(s) to the current version: [gordonr@smebuild smeserver-yum]$ ls P
 
CVS
 
smeserver-yum-1.2.0-DisplayStatus.patch
 
smeserver-yum-1.2.0-DisplayStatus.patch2
 
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
 
* The <tt class="FILENAME">S</tt> directory contains the source tarball(s): [gordonr@smebuild smeserver-yum]$ ls S
 
CVS  smeserver-yum-1.2.0.tar.gz
 
 
   
 
   
 +
chmod 600 ~/.ssh/config
  
<div class="NOTE"><blockquote class="NOTE">
+
====Import source to shell.koozali.org====
 +
Email admin@contribs.org with the location of your rpm, it will be imported into the build system for you. Follow the same procedure when an upstream release occurs, eg a new .tar.gz, Update your local cvs with:
  
'''Note: '''The <tt class="FILENAME">smeserver-yum</tt> subdirectory is a leftover from an incorrect package import. It can be ignored and will be removed when maintenance is next performed on the CVS repository.
+
cvs update -dPA
  
</blockquote></div></div><div class="SECT2">
+
If you have developer access to buildsys you can follow this guide
----
 
  
===mzclean: Clean out modified files===
+
https://wiki.contribs.org/Package_Import
  
Before making changes to a package, you should ensure that you have a clean copy of the latest sources. Remember - other people may also be working on the package and making changes in SourceForge. You can perform a "merge" when you check in your changes, but it is easiest to start with an up-to-date copy. It is even possible that someone else has already fixed the bug you are working on!
+
====Import cvs in your workspace====
 +
You can use ~/home/smeserver or whatever suits.
  
  [gordonr@smebuild smeserver-yum]$ mzclean
+
  mkdir ~/home/smeserver
  Cleaning and resyncing smeserver-yum, please wait....
+
  cd ~/home/smeserver
  Removing work....
+
  cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P rpms
Cleanup of smeserver-yum complete.
 
 
[gordonr@smebuild smeserver-yum]$ ls
 
ChangeLog  CVS  F  P S
 
  
<div class="TIP"><blockquote class="TIP">
+
mkdir ~/home/smecontribs
 
+
cd ~/home/smecontribs
'''Tip: '''Ensure that you have copied any modified files out of your work tree prior to runing '''mzclean'''.
+
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P rpms
 
 
</blockquote></div>
 
 
 
Note that the stray <tt class="FILENAME">smeserver-yum</tt> subdirectory has been removed.
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===mzprep: Build working tree===
 
 
 
The '''mzprep''' command runs '''rpmbuild -bp''' on your SPEC file, sources and patches to generate a working tree. You can make changes in the working tree and use mezzanine to generate the patches between the previous version and your new files.
 
 
 
[gordonr@smebuild smeserver-yum]$ mzprep
 
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
 
You may now chdir to work to make changes.
 
Use "mzpatch -n <patch_name>" to generate a patch when done.
 
 
   
 
   
  [gordonr@smebuild smeserver-yum]$ ls
+
To refresh run the following from the rpms directory, or any lower directory with a CVS dir
ChangeLog  CVS  F  P  S  work
+
  cvs update -dPA
  
The only obvious change is the creation of a <tt class="FILENAME">work/</tt> subdirectory. Let's have a look in there. Under the <tt class="FILENAME">work/</tt> directory is <tt class="FILENAME">smeserver-yum-1.2.0/</tt> which contains the <span class="emphasis">''patched''</span> sources for this version of <var class="LITERAL">smeserver-yum</var>.
+
</div></div><div class="SECT1">
 
 
[gordonr@smebuild smeserver-yum]$ ls work/
 
smeserver-yum-1.2.0
 
 
 
Under <tt class="FILENAME">smeserver-yum-1.2.0/</tt> are the <tt class="FILENAME">createlinks</tt> script, the <tt class="FILENAME">root</tt> hierarchy and a few generated files, explained below.
 
 
 
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/
 
createlinks      debugsources.list  smeserver-yum-1.1.2-26-filelist
 
debugfiles.list  root
 
 
 
</div><div class="SECT2">
 
 
----
 
----
  
===SME Server package directory layout===
+
===Modifying a SME Server package===
  
<div class="TABLE">
+
<div class="SECT2">
  
'''Table 14-1. SME Server package directory layout'''
+
====Raise a Bugzilla entry====
  
{| class="CALSTABLE" border="1"
+
Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug <span class="emphasis">''before''</span> you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.
! Directory/File
 
! Description
 
| createlinks
 
| Builds action, panel and initscript links.
 
|-
 
| po
 
| Package translations, if gettext is used.
 
|-
 
| root
 
| The top-level directory of the package, which will be the <tt class="FILENAME">/</tt> directory of the installed system. So, <tt class="FILENAME">root/usr/lib/perl5/site_perl</tt> becomes <tt class="FILENAME">/usr/lib/perl5/site_perl</tt> on the installed system. Note that the somewhat confusing <tt class="FILENAME">root/root</tt> is the <tt class="FILENAME">/root</tt> directory of the installed system.
 
|-
 
| update-po
 
| Helper script to generate translation binary files, if gettext is used.
 
|-
 
| debugsources.list debugfiles.list
 
| Generated by RPM build - lists this files which are built with debug flags
 
|-
 
| smeserver-yum-1.1.2-26-filelist
 
| Generated by RPM build - lists the files to be packaged, and their permissions and ownership
 
|}
 
 
 
</div></div><div class="SECT2">
 
----
 
  
===The root directory hierarchy===
+
<span class="emphasis">''All changes must have an associated Bugzilla entry.''</span> The bug tracker is here: http://bugs.contribs.org/
  
The <tt class="FILENAME">work/smeserver-yum-1.2.0/root/</tt> hierarchy contains the files which will be installed on the target machine.
+
If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:
  
[gordonr@smebuild smeserver-yum]$ ls work/smeserver-yum-1.2.0/root/
+
For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" [http://bugs.contribs.org/show_bug.cgi?id=1174 http://bugs.contribs.org/show_bug.cgi?id=1174].
etc  sbin  service  usr  var
 
  
 
</div><div class="SECT2">
 
</div><div class="SECT2">
 
----
 
----
  
===Modifying a file===
+
====Modify the package====
  
OK - now we can actually make a change. We want to modify the file which will be installed as <tt class="FILENAME">/etc/e-smith/events/actions/yum-import-keys</tt>, so the actual file we need to modify is: <tt class="FILENAME">work/smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys</tt>.
+
If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run '''rpm -qf''' on the file to be modified:
  
Use your favourite editor (in Unix text mode) to modify the file, adding a comment where the change needs to be made. Here's a '''diff''' showing such a comment:
+
[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys
 +
smeserver-yum-1.1.2-05
  
[gordonr@smebuild tmp]$ diff -u yum-import-keys.orig yum-import-keys
+
and so, we want to modify the <tt class="FILENAME">smeserver-yum</tt> package.
--- yum-import-keys.orig        2006-05-22 16:28:51.734609534 +1000
 
+++ yum-import-keys    2006-05-22 16:28:39.006814950 +1000
 
@@ -30,6 +30,7 @@
 
  for my $key ( grep {!/^\./} readdir(DIR) )
 
  {
 
      warn "Importing key $key\n";
 
+    # TODO: Skip keys which have previously been imported
 
 
      system("rpm", "--import", $key) == 0 or
 
        warn "Couldn't rpm --import $key\n";
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===mzpatch: Create a patch===
 
 
 
Mezzanine automates the creation of patches through the '''mzpatch''' command.
 
 
 
<div class="NOTE"><blockquote class="NOTE">
 
 
 
'''Note: '''Always run '''mzpatch''' from the top-level directory of your package (e.g. <tt class="FILENAME">~/smeserver/smeserver-yum/</tt>).
 
 
 
</blockquote></div>
 
 
 
[gordonr@smebuild actions]$ cd ~/smeserver/smeserver-yum/
 
 
[gordonr@smebuild smeserver-yum]$ ls P
 
CVS
 
smeserver-yum-1.2.0-DisplayStatus.patch
 
smeserver-yum-1.2.0-DisplayStatus.patch2
 
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
 
 
 
You will see from the listing above that the patches should be named <span class="emphasis">''package-version-patchname.patch''</span>. You should also ensure that the patch name you are about to choose doesn't already exist, or you may overwrite an older version of the patch.
 
  
[gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
+
<span class="emphasis">''All packages on the SME Server ISO/CD must be checked into shell.koozali.org CVS''</span>. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.
Creating working directory /home/e-smith/files/users/gordonr/tmp/sf/smeserver-yum/work....
 
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
 
cvs [server aborted]: "add" requires write access to the repository
 
You do not have write access to the master repository.
 
srctool:  Error:  Addition of P/smeserver-yum-1.2.0-ImportKeysComment.patch failed.
 
[gordonr@smebuild smeserver-yum]$ ls P
 
CVS
 
smeserver-yum-1.2.0-DisplayStatus.patch
 
smeserver-yum-1.2.0-DisplayStatus.patch2
 
smeserver-yum-1.2.0-ImportKeysComment.patch
 
smeserver-yum-1.2.0-ModifyUpdateDBs.patch
 
 
 
Mezzanine has created the patch, but it has not committed it to CVS. This allows you to revise the patch until it works, prior to committing it. If you do not have a developer account you will get an error when mezzanine attempts to reserve the patch name in CVS, but the patch will still be created in the <tt class="FILENAME">P</tt> directory.
 
 
 
The patch looks quite similar to the one we saw previously, except for the pathname to the modified file:
 
 
 
[gordonr@smebuild smeserver-yum]$ cat P/smeserver-yum-1.2.0-ImportKeysComment.patch
 
diff -Nur -x '*.orig' -x '*.rej' smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys
 
--- smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys 2005-09-26 12:30:54.000000000 +1000
 
+++ mezzanine_patched_smeserver-yum-1.2.0/root/etc/e-smith/events/actions/yum-import-keys      2006-05-22 16:35:57.752811633 +1000
 
@@ -30,6 +30,7 @@
 
  for my $key ( grep {!/^\./} readdir(DIR) )
 
  {
 
      warn "Importing key $key\n";
 
+    # TODO: Skip keys which have previously been imported
 
 
      system("rpm", "--import", $key) == 0 or
 
        warn "Couldn't rpm --import $key\n";
 
  
 
</div><div class="SECT2">
 
</div><div class="SECT2">
 
----
 
----
  
===Building a new package, with the patch===
+
You can now retrieve one of the packages from shell.koozali.org. In this case, we want to modify the <var class="LITERAL">smeserver-yum</var> package, so let's retrieve it from shell.koozali.org:
  
The next step is to change the package <tt class="FILENAME">SPEC</tt> file so that the patch is applied.
+
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P smeserver-yum
  
<div class="SECT3">
+
Change to work directory
----
+
cd smeserver/rpms/smeserver-yum/sme7
  
====Update the Release tag====
+
To prepare a tree
 +
cvs update -dPA 
 +
make clean       
 +
make prep
  
The <span class="emphasis">''first''</span> change you should make is to update the "Release" tag in the <tt class="FILENAME">SPEC</tt> file. You should do this by appending your initials and sequence number to the <var class="LITERAL">Release</var> number. In most SME Server <tt class="FILENAME">SPEC</tt> files, the <var class="LITERAL">Release</var> number is set via the <var class="LITERAL">release</var> macro, and so you should set it there.
+
=====Make a patch=====
 +
Then switch to the tree and make modification.
  
Summary: YUM, an rpm updater
+
Method A
%define name smeserver-yum
 
Name: %{name}
 
%define version 1.2.0
 
- %define release 05
 
+ %define release 05ff01
 
  
Please just append your initials and a sequence number to the release tag. Do not change the package name (e.g. to smeserver-yum-fred) or the version number. This policy makes it obvious that your update is a patch based on <var class="LITERAL">smeserver-yum-1.2.0-05</var>.
+
In the prepared dir copy a file you want to modify like so:
 
 
If a change has been approved by the Development Manager, it is given an official version/release number (e.g. smeserver-yum-1.2.0-06). <span class="emphasis">''Always tag unofficial changes with your initials and sequence number.''</span>
 
 
 
If an upstream (e.g. CentOS) package needs to be patched and it is not possible to wait for the patch to come from the upstream source, the release number should be modified with the special tag <var class="LITERAL">sme</var> and a sequence number (e.g. clamav-0.88.2-1sme01). <span class="emphasis">''The sme tag is reserved for changes approved by the Development Manager.''</span>
 
 
 
</div><div class="SECT3">
 
----
 
  
====Add a changelog entry====
+
cp yum-import-keys yum-import-keys.{patchname}
  
The SME Server project maintains history of packages changes in the <var class="LITERAL">%changelog</var> section of the <tt class="FILENAME">SPEC</tt> file. The changelog entry should be a simple description of the change of behaviour and must include a reference to an SME Server project Bugzilla entry (1174) and the new version number (1.2.0-05ff01):
+
Then modify the original file yum-import-keys.
 +
To add new files touch yum-import-keys.{patchname} so it is empty.
  
<nowiki>%changelog
+
Once you have all the files you want patched copied and changed then you can build the patch (from the sme7 dir) with:
+ * Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
+
  make patch SUFFIX={patchname}
+ - Initial work on avoiding duplicate RPM keys [SME: 1174]
+
It will build and add the patch for you. It should be named "name-version-{patchname}.patch". It will also add the patch to CVS.
   
 
  * Mon May  1 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
 
- Remove stray yum.pm.orig file. [SME: 1350]</nowiki>
 
  
The first line of the changelog entry shows when the change was made, by whom and notes the new version number. The other lines describe the change and refer to the Bugzilla entry or entries, using the format <var class="LITERAL">[SME: 1174]</var> as a shorthand.
+
Method B
  
</div><div class="SECT3">
+
Make a copy of the prepared directory, edit directly, then make a patch
----
 
  
====Add a patch reference====
+
cp -R smeserver-yum-2.0.0 smeserver-foo-2.0.0.old
 +
diff -urN smeserver-foo-2.0.0.old smeserver-yum-2.0.0 > smeserver-yum-2.0.0-importKeys.patch
  
You now need to tell RPM to retrieve and apply the new patch. This needs to be done in two places, which is annoying, but allows for patches to be carried around, but not applied (e.g. because they are not quite finished). The first change is near the top of the file:
+
=====Apply a patch=====
 +
For example, check if a translation patch is available:
 +
* for [http://translate.contribs.org/patches/ SME Server base]
 +
* for [http://translate.contribs.org/patches/contribs/ SME Server contribs]
  
Patch0: smeserver-yum-1.2.0-DisplayStatus.patch
+
If patch size is 0 bytes there is nothing to do
Patch1: smeserver-yum-1.2.0-ModifyUpdateDBs.patch
 
Patch2: smeserver-yum-1.2.0-DisplayStatus.patch2
 
+ Patch3: smeserver-yum-1.2.0-ImportKeysComment.patch
 
  
The second change is in the <var class="LITERAL">%prep</var> section:
+
Else go to the package folder in your tree and do :
  
  %prep
+
  wget http://translate.contribs.org/patches/contribs/{name}-locale-{date}.patch
%setup
+
  cvs add {name}-locale-{date}.patch
%patch0 -p1
 
%patch1 -p1
 
%patch2 -p1
 
rm root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/yum.pm.orig
 
  + %patch3 -p1
 
  
</div><div class="SECT3">
+
Then you need to follow instructions in next part....
----
 
  
====mzbuild: Build a new package====
+
You may add yourself some [http://translate.contribs.org/ translations ], and wait for patch to be created (at about 2 AM  GMT-6, or 6 PM Sydney)
  
With all that done, you should be able to build a new package using '''mzbuild'''<nowiki>:</nowiki>
+
=====Edit the spec=====
 +
nano -w smeserver-foo.spec
  
  [gordonr@smebuild smeserver-yum]$ mzbuild
+
  #increase the release
 +
%define release 15
 
   
 
   
  [...]
+
  #add the patch
  Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.35037
+
  Patch2: smeserver-foo-1.2-widget.patch
+ umask 022
 
+ cd /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/build.mezz/BUILD
 
+ cd smeserver-yum-1.2.0
 
+ /bin/rm -rf /var/tmp/mezzanine-buildroot.18508
 
+ exit 0
 
Package build succeeded.  Output files are:
 
 
   
 
   
  smeserver-yum-1.2.0-05ff01.src.rpm
+
  #update the changelog, include the bug number
smeserver-yum-1.2.0-05ff01.noarch.rpm
+
* Fri Jan 11 2008 John Smith <smith@foo.net> 1.2-15
 
+
  - fixed foo to create bar [SME 3470]
</div><div class="SECT3">
 
----
 
 
 
====Check the package====
 
 
 
You should now perform some simple checks on the package before installing it for testing.
 
 
 
* Does the built package have your initials in the release tag? We never want to have two packages with the same name and version number. [gordonr@smebuild smeserver-yum]$ ls *.rpm
 
smeserver-yum-1.2.0-05ff01.noarch.rpm smeserver-yum-1.2.0-05ff01.src.rpm
 
 
   
 
   
* Did you update the changelog? It's easy to forget, but vital to do. <nowiki>[gordonr@smebuild smeserver-yum]$ rpm -qp --changelog \
+
  #apply the patch in %setup
  smeserver-yum-1.2.0-05ff01.noarch.rpm | head -8
+
  %patch2 -p1
   
 
* Mon May 22 2006 Fred Frog <fred@example.com> 1.2.0-05ff01
 
 
- Initial work on avoiding duplicate RPM keys [SME: 1174]
 
 
* Mon May 01 2006 Charlie Brady <charlieb@e-smith.com> 1.2.0-05
 
 
- Remove stray yum.pm.orig file. [SME: 1350]
 
  </nowiki>
 
* Did you remember to apply the patch? It happens, and leads to head scratching "I knew I made that change". [gordonr@smebuild smeserver-yum]$ mzbuild | grep patch
 
  + echo 'Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):'
 
Patch #0 (smeserver-yum-1.2.0-DisplayStatus.patch):
 
+ patch -p1 -s
 
+ echo 'Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):'
 
Patch #1 (smeserver-yum-1.2.0-ModifyUpdateDBs.patch):
 
+ patch -p1 -s
 
+ echo 'Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):'
 
Patch #2 (smeserver-yum-1.2.0-DisplayStatus.patch2):
 
+ patch -p1 -s
 
+ echo 'Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):'
 
Patch #3 (smeserver-yum-1.2.0-ImportKeysComment.patch):
 
+ patch -p1 -s
 
 
 
 
</div></div><div class="SECT2">
 
----
 
 
 
===Test your new package===
 
 
 
You should now upgrade your test system:
 
 
 
[gordonr@sevendev1 tmp]$ sudo rpm -Uvh smeserver-yum-1.2.0-05ff01.noarch.rpm
 
Preparing...                ########################################### [100%]
 
    1:smeserver-yum          ########################################### [100%]
 
 
 
and test that you have fixed the bug.
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===Lather, rinse, repeat===
 
  
If you haven't fixed the bug, repeat the cycle: make changes, build a new patch, update the release number (e.g. to 05ff02, 05ff03, etc.), build a new package, test. Don't worry about using up release numbers.
+
=====Commit=====
 +
Build the rpm locally to test, (note, this deletes the working tree!)
 +
make local     
  
<div class="NOTE"><blockquote class="NOTE">
+
Once you are satisfied and want to submit the package to the build server commit your changes. (Please use descriptive comments so that other developers are aware of what is happening.  Comments will appear on the subject line of the commit email that get send to the other developers.)
  
'''Note: '''Actually fixing bug 1174 is left as an exercise for the reader. Fame and fortune (or at least thanks) await the person who does.
+
cvs commit -m 'your descriptive commit message here'
  
</blockquote></div></div><div class="SECT2">
+
You can automate the addition of the comments in the spec file with the command 'clog'. {{Note box|You will need to be in the sme7, sme8, or contribs7 directory for this to work}}
----
 
  
===Attach your patch(es) to the bug===
+
rm -f clog
 +
cvs commit -m "$(make clog)"
  
You should now attach your patch(es) and SPEC file changes to the appropriate Bugzilla entry. If the change is the simple application of a patch, you do not need to attach the whole SPEC file - just the changelog entry.
+
CVS cheat sheet [[:Package_Modification/More cvs commands]]
 
 
Please attach the patch(es) and ensure that they are in Unix text format. Patches make it obvious which sections of code have been changed and attaching them ensures that their formatting is preserved.
 
 
 
</div></div><div class="SECT1">
 
----
 
  
==SourceForge developer CVS access==
+
=====Build=====
 +
Tag all files as belonging to a particular build version
 +
make tag
  
If you are going to work on many bugs, or wish to help with the many tasks involved in a release, you should create a SourceForge developer account.
+
Submit the request to the build server which will checkout the recently tagged version and build it
 +
make build
  
<span class="emphasis">''You do not need a SourceForge account in order to do development.''</span> You can contribute very effectively by producing patches and attaching them to the Bugzilla entries.
+
Always do "make tag" before "make build"  
  
<div class="SECT2">
+
Always ensure you are working with the latest version (cvs update -dPA)
----
 
  
===Create a SourceForge account===
+
You and updatesteam will get an email on successful build. 
 +
Only you will get an email on failed build.
  
* Create a SourceForge user id
+
You can check the build system is working: 
* Contact the Development Manager (or one of the other project admins) listed on the SourceForge project page, detailing the packages you would like to work on
 
* The project admin will add you as a developer
 
* Upload an SSH2 key to SourceForge, following the SourceForge documentation: https://sourceforge.net/docs/F02/en/#key_posting
 
* You will also need to generate another SSH2 key for access to the smebuild build server
 
  
</div><div class="SECT2">
+
* [https://buildsys.contribs.org/plague https://buildsys.contribs.org/plague]
----
+
* or in shell
 +
plague-client list uid  {task number}
  
===mzgetsf: SourceForge developer CVS checkout===
+
====Releasing a contrib package====
 +
After the make build command the build system will try and build your package. After a successful build it will be put in the smetest repository. You should be notified of the result of the build by e-mail.
  
Once you have SourceForge CVS access, you can use '''mzgetsf''' (instead of '''mzgetsf_anon''') to retrieve the sources. Your copy will be writable and you will be able to put changes back to CVS. When you run '''mzpatch''' you should not see an error during the '''cvs add''' step.
+
Once a package is build successfully you should verify your changes, ideally you would have a bug to verify for each modification. After verification of all relevant changes and bugs you can release the package like this:
 +
<ol></li><li>Login to shell.contribs.org like this:
  
  [gordonr@smebuild smeserver-yum]$ mzpatch -n smeserver-yum-1.2.0-ImportKeysComment.patch
+
  ssh -p222 username@shell.koozali.org
Creating working directory /home/e-smith/files/users/gordonr/smeserver/smeserver-yum/work....
+
</li><li>Navigate to the smeserver directory:
Created P/smeserver-yum-1.2.0-ImportKeysComment.patch (11 lines).
 
cvs add: use 'cvs commit' to add this file permanently
 
Patch added to SCM.  Use 'mzput' to upload to repository.
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===mzput: Put the changes back to CVS===
 
 
 
Instead of attaching the patch to the Bugzilla entry, you will be able to put it back to CVS. Mezzanine normally edits the <tt class="FILENAME">ChangeLog</tt>, but we don't need to do that as we keep the changes in the <var class="LITERAL">%changelog</var> section of the <tt class="FILENAME">SPEC</tt> file. The <var class="LITERAL">-m</var> option to '''mzput''' allows a simple comment to be appended, which is all that is required:
 
 
 
[gordonr@smebuild smeserver-yum]$ mzput -m 'See changelog'
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===Build the official package===
 
 
 
Official packages have unadorned release tags (e.g. smeserver-yum-1.2.0-05) and must be built on the official build servers. There are three steps involved - build the RPMs ('''mzbuild'''), sign the RPMs ('''rpm --addsign *.rpm''') and release the RPMs ('''release_rpms *.rpm''').
 
 
 
</div><div class="SECT2">
 
----
 
 
 
===mzimportsf: SourceForge package import===
 
 
 
If a package has not already been checked into SourceForge CVS, you can use '''mzimportsf''' to import it. You should be careful to run this command from a directory which does not contain an imported package or mezzanine may import the package underneath the existing package.
 
 
 
</div><div class="SECT2">
 
----
 
  
===mzmerge: Merge changed source RPM===
+
cd /teams/smeserver
 +
</li><li>The teams directory contains a few directories of which two are relevant, the first is called updates which will hold the SME Server packages, the other is called contribs and will hold build contribs.
 +
Suppose we would like to release our contrib we would proceed like this:
  
If a new source RPM has been built without corresponding changes in SourceForge CVS, you can use '''mzmerge''' to merge in the changes. This can also be used to merge in changes from a modified upstream package. Care should be exercised with this command, especially if you want to maintain previous patches.
+
cd contribs/9
 +
</li><li>Now copy the relevant package from smetest to smecontribs, old versions are removed automatically
  
</div></div></div></div><div class="PART"><div class="TITLEPAGE">
+
cp smetest/package-name-version.rpm smecontribs/
 +
</li></ol>
  
=IV. Advanced customization of the SME Server=
+
{{Note box|Within a period of one hour the package should be moved to the smecontribs repository and be available as soon as the mirrors synchronize.}}
  
<div class="TOC">
+
Once the server successfully builds it will automatically be pulled on the next repo update run
 +
(40 past the even hours MDT). The package will either be put into the smedev (new package) or smetest (exist in higher repo)  After verification the package is manually moved from smedev/smetest to smecontribs (for contribs)  or smeupdates-testing (for packages in base)
  
; '''Table of Contents'''
+
===Mailing Lists===
; 15. [http://wiki.contribs.org/The_SME_Server_Developer%27s_Guide#ADVANCED-CUSTOMIZATION Advanced customization principles]
+
Subscribe yourself to the [http://lists.contribs.org/mailman/listinfo/devinfo devinfo mailinglist]. This is the place to discuss the development of the server and contribs. If you have other questions, not regarding development please use the [http://forums.contribs.org forums].
  
 
</div></div>
 
</div></div>

Latest revision as of 01:59, 12 March 2020

The SME Server development environment

Configuring your development environment

The SME Server source code is checked into CVS at koozali.org. SME Server code is stored in the CVS on shell.koozali.org in two repositories:

Reminder: The SME Server source code is released under the GPL. You must release the source code to all modifications. If you make improvements, please raise a bug and attach a patch so the change can be discussed and pulled back into the base for everyone to share.

Only developers who are going to put patches back into CVS and build new packages need shell.koozali.org CVS access. The sources are freely available and patches are gratefully received. Just follow the instructions in this section and attach the patch(es) to the Bugzilla entry, explaining why the change should be made.


Local environment

  • Install cvs
yum install cvs rsh
  • Setup CVS to use ssh by creating /etc/profile.d/smebuild.sh with the following content
# Developer environment
# This gets symlinked into /etc/profile.d 

export CVS_RSH=ssh # tell CVS to use ssh

# DO NOT set CVSROOT

alias rm='rm -i'
alias cp='cp -i --preserve=timestamps'
alias mv='mv -i'

You have to logout and login again to the console for changes to take effects.

Access to build system

Check updates/status on the build server: http://buildsys.contribs.org


Warning.png Warning:
For contrib builders using the plague-client-0.5.0 you will need to downgrade to correct version, to fix this please do:
rpm -e plague-client plague-common
yum --enablerepo=smecontribs install plague-client

Ask admin@contribs.org for certificates, give the email address to use for notifications.

yum --enablerepo=smecontribs install plague-client

From now, do not use account "root" anymore. Use a dedicated dev account. Copy certificates and config file to ~/

.plague-client.cfg
.username.pem
.contribs-upload-ca.pem
.contribs-server-ca.pem

Don't forget to set the proper privileges on the file

chmod 600 .username.pem

Check it's working

plague-client list_builders

Builders:
------------------------------------------------------------------------------------------
 build64-1.contribs.org      x86_64 amd64 ia32e noarch i386 i486 i586 i686 athlon available
 build32-1.contribs.org      i386 i486 i586 i686 athlon noarch  available

CVS shell.koozali.org access

  • SME Server code is stored in the CVS on http://shell.koozali.org. To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org. With this account the development team can give you access to the CVS repository.
  • After your Koozali.org account has been created you can ask the development team to give you developer access to smecontribs. Create a bug in the Bug Tracker as usual.
  • If local username is different to shell.koozali.org username edit ~/.ssh/config:
Host  shell.koozali.org
User  koozaliusername (without @shell.koozali.org)
port 222
  • Don't forget to set the proper privileges on the file
chmod 600 ~/.ssh/config

Import source to shell.koozali.org

Email admin@contribs.org with the location of your rpm, it will be imported into the build system for you. Follow the same procedure when an upstream release occurs, eg a new .tar.gz, Update your local cvs with:

cvs update -dPA

If you have developer access to buildsys you can follow this guide

https://wiki.contribs.org/Package_Import

Import cvs in your workspace

You can use ~/home/smeserver or whatever suits.

mkdir ~/home/smeserver
cd ~/home/smeserver
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P rpms 
mkdir ~/home/smecontribs
cd ~/home/smecontribs 
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P rpms 

To refresh run the following from the rpms directory, or any lower directory with a CVS dir

cvs update -dPA

Modifying a SME Server package

Raise a Bugzilla entry

Before you make any changes to a package, you need to have a Bugzilla entry which specifies the problem and preferably proposes a fix. Raising the bug before you do the work allows others to comment on the proposed approach and can save significant time when you go to submit the changes. The change should also be approved by the Development Manager if it is meant for near-term release. You will need the Bugzilla bugid when you check in the changes.

All changes must have an associated Bugzilla entry. The bug tracker is here: http://bugs.contribs.org/

If a relevant bug does not exist, raise one. If the bug exists, assign it to yourself to show that you are working on it:

For this exercise, let's look at bug 1174 "yum-import-keys should not import duplicates" http://bugs.contribs.org/show_bug.cgi?id=1174.


Modify the package

If you are modifying an existing file, the simplest way to determine the package is to install the relevant version and run rpm -qf on the file to be modified:

[gordonr@smebuild actions]$ rpm -qf /etc/e-smith/events/actions/yum-import-keys
smeserver-yum-1.1.2-05

and so, we want to modify the smeserver-yum package.

All packages on the SME Server ISO/CD must be checked into shell.koozali.org CVS. The only exceptions are packages which come from the following upstream repositories: CentOS and dag.


You can now retrieve one of the packages from shell.koozali.org. In this case, we want to modify the smeserver-yum package, so let's retrieve it from shell.koozali.org:

cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P smeserver-yum

Change to work directory

cd smeserver/rpms/smeserver-yum/sme7

To prepare a tree

cvs update -dPA   
make clean        
make prep
Make a patch

Then switch to the tree and make modification.

Method A

In the prepared dir copy a file you want to modify like so:

cp yum-import-keys yum-import-keys.{patchname}

Then modify the original file yum-import-keys. To add new files touch yum-import-keys.{patchname} so it is empty.

Once you have all the files you want patched copied and changed then you can build the patch (from the sme7 dir) with:

make patch SUFFIX={patchname}

It will build and add the patch for you. It should be named "name-version-{patchname}.patch". It will also add the patch to CVS.

Method B

Make a copy of the prepared directory, edit directly, then make a patch

cp -R smeserver-yum-2.0.0 smeserver-foo-2.0.0.old
diff -urN smeserver-foo-2.0.0.old smeserver-yum-2.0.0 > smeserver-yum-2.0.0-importKeys.patch
Apply a patch

For example, check if a translation patch is available:

If patch size is 0 bytes there is nothing to do

Else go to the package folder in your tree and do :

wget http://translate.contribs.org/patches/contribs/{name}-locale-{date}.patch
cvs add {name}-locale-{date}.patch

Then you need to follow instructions in next part....

You may add yourself some translations , and wait for patch to be created (at about 2 AM GMT-6, or 6 PM Sydney)

Edit the spec
nano -w smeserver-foo.spec
#increase the release
%define release 15

#add the patch
Patch2: smeserver-foo-1.2-widget.patch

#update the changelog, include the bug number
* Fri Jan 11 2008 John Smith <smith@foo.net> 1.2-15
- fixed foo to create bar [SME 3470]

#apply the patch in %setup
%patch2 -p1
Commit

Build the rpm locally to test, (note, this deletes the working tree!)

make local       

Once you are satisfied and want to submit the package to the build server commit your changes. (Please use descriptive comments so that other developers are aware of what is happening. Comments will appear on the subject line of the commit email that get send to the other developers.)

cvs commit -m 'your descriptive commit message here'

You can automate the addition of the comments in the spec file with the command 'clog'.

Important.png Note:
You will need to be in the sme7, sme8, or contribs7 directory for this to work

rm -f clog
cvs commit -m "$(make clog)"

CVS cheat sheet Package_Modification/More cvs commands

Build

Tag all files as belonging to a particular build version

make tag

Submit the request to the build server which will checkout the recently tagged version and build it

make build

Always do "make tag" before "make build"

Always ensure you are working with the latest version (cvs update -dPA)

You and updatesteam will get an email on successful build. Only you will get an email on failed build.

You can check the build system is working:

plague-client list uid  {task number}

Releasing a contrib package

After the make build command the build system will try and build your package. After a successful build it will be put in the smetest repository. You should be notified of the result of the build by e-mail.

Once a package is build successfully you should verify your changes, ideally you would have a bug to verify for each modification. After verification of all relevant changes and bugs you can release the package like this:

  1. Login to shell.contribs.org like this: ssh -p222 username@shell.koozali.org
  2. Navigate to the smeserver directory: cd /teams/smeserver
  3. The teams directory contains a few directories of which two are relevant, the first is called updates which will hold the SME Server packages, the other is called contribs and will hold build contribs. Suppose we would like to release our contrib we would proceed like this: cd contribs/9
  4. Now copy the relevant package from smetest to smecontribs, old versions are removed automatically cp smetest/package-name-version.rpm smecontribs/


Important.png Note:
Within a period of one hour the package should be moved to the smecontribs repository and be available as soon as the mirrors synchronize.

Once the server successfully builds it will automatically be pulled on the next repo update run (40 past the even hours MDT). The package will either be put into the smedev (new package) or smetest (exist in higher repo) After verification the package is manually moved from smedev/smetest to smecontribs (for contribs) or smeupdates-testing (for packages in base)

Mailing Lists

Subscribe yourself to the devinfo mailinglist. This is the place to discuss the development of the server and contribs. If you have other questions, not regarding development please use the forums.