Updates cycles
This page is in a Draft stage and still requires additions and clarifications.
OVERVIEW
The SME Server update and maintenance cycle consists of the management of a number of repositories and package categories.
In order to help in the release cycle, mainly on pushing upstream packages at the right place, you can refer with some caution to this script : http://canada.pialasse.com/SMErepo/
Repositories
The master directories associated with the update and maintenance cycles are located on the build system. Their content is then propagated to the mirrors.
They are:
- dev [corresponds to smedev in the mirrors]
- test [corresponds to smetest in the mirrors]
- updates-testing [corresponds to smeupdates-testing in the mirrors]
- updates [corresponds to smeupdates in the mirrors]
In addition, there is a staging area (stage) that is used to build the next ISO, you will see reference to it in some repository update emails.
Package Categories
Sme modified packages.
In the context of the updates cycles, these are packages found in smetest, smeupdates-testing or smeupdates for which an older package can be found in smeos (To be confirmed). Changelog generated in-house and includes a reference to a Bug in Bugzilla. Verification required before release.
Upstream packages
i.e. packages from centos, epel, atrpms, rpmforge, etc). These are either not SME modified packages, or are kernel mods. There are no bug reports or in-house changelog generated, summary testing only before release.
Clamav packages.
This is a special case. Whilst initially not build by SME, and due to problem having been experienced in the past, these packages (three of them) are attached to a single SME bug in bugzilla. There is no specific changelog, however verification is recommended before release.
Initscript packages.
This is a special case up to SME9 (including). Rebuild by SME every time upstream release a higher version to what we built. We import the new package in CVS and apply our patches.
so first import new srpm
cd smeserver/rpms/initscripts/sme9
wget http://vault.centos.org/
../../common/cvs-import.sh -b sme9 -m "import initscript from centOS" initscripts-.....el6.centos.src.rpm
second restore needed patches
rm -f initscripts-9.03.31-runlevel7.patch initscripts-9.03.40-dmesg.patch initscripts-9.03.40-slapd_alias_ldap.patch initscripts-9.03.46-dont_run_plymouth_if_disabled.patch
cvs update -dPA
cvs update -p -r 1.2 initscripts-9.03.31-runlevel7.patch > initscripts-9.03.31-runlevel7.patch
cvs add initscripts-9.03.31-runlevel7.patch
cvs update -p -r 1.1 initscripts-9.03.40-dmesg.patch > initscripts-9.03.40-dmesg.patch
cvs add initscripts-9.03.40-dmesg.patch
cvs update -p -r 1.1 initscripts-9.03.40-slapd_alias_ldap.patch > initscripts-9.03.40-slapd_alias_ldap.patch
cvs add initscripts-9.03.40-slapd_alias_ldap.patch
cvs update -p -r 1.1 initscripts-9.03.46-dont_run_plymouth_if_disabled.patch > initscripts-9.03.46-dont_run_plymouth_if_disabled.patch
cvs add initscripts-9.03.46-dont_run_plymouth_if_disabled.patch
For SME the patch to add in spec file:
Patch2000: initscripts-9.03.31-runlevel7.patch Patch2001: initscripts-9.03.40-dmesg.patch Patch2002: initscripts-9.03.40-slapd_alias_ldap.patch Patch2003: initscripts-9.03.46-dont_run_plymouth_if_disabled.patch
in %setup we add :
%patch2000 -p1 %patch2001 -p1 %patch2002 -p1 %patch2003 -p1
then update the changelog,
%changelog
* date your name <email> - version release
- Rebase on upstream ...'''version'''... [SME '''...''']
- Don't try to run plymouth if disabled [SME: 8375]
Code from Charlie Brady
- Make slapd service an alias for ldap [SME: 8635]
- Add -n 1 to the dmesg line in rc.sysinit to prevent unwanted
messages appearing on the console [SME: 8277]
- Add hack for running rc7.d script during runlevel 4 [SME: 7217]the release version is the same as upstream and we add trailing release number :
Release: 1%{?dist}.1
If you are lucky the previous patches will still work, otherwise you will need to manually modify and create new patches.
Samba
As per SME 10 we build our own version of Samba from upstream in order to enable few functions to allow Samba AD usage. All modifications are done in the spec file.
the changes are the following :
--- samba.spec.original 2016-10-02 16:23:04.356835288 -0700
+++ samba.spec 2016-10-02 16:31:38.311833030 -0700
@@ -38,1 +38,1 @@
-%global with_internal_ldb 0
+%global with_internal_ldb 1
@@ -59,8 +59,8 @@
%global libwbc_alternatives_suffix -64
%endif
-%global with_mitkrb5 1
-%global with_dc 0
+%global with_mitkrb5 0
+%global with_dc 1
%if %{with testsuite}
# The testsuite only works with a full build right now.
@@ -399,2 +399,3 @@
Summary: Samba libraries
Group: Applications/System
+Requires: libldb
@@ -463,7 +464,7 @@
Requires: %{name}-libs = %{samba_depver}
Requires: python-tevent
Requires: python-tdb
-Requires: pyldb
+#Requires: pyldb
Requires: pytalloc
Provides: samba4-python = %{samba_depver}
@@ -864,8 +865,8 @@
%endif
install -d -m 0755 %{buildroot}%{_unitdir}
-for i in nmb smb winbind ; do
- cat packaging/systemd/$i.service | sed -e 's@\[Service\]@[Service]\nEnvironment=KRB5CCNAME=FILE:/run/samba/krb5cc_samba@g' >tmp$i.service
+for i in nmb smb winbind samba; do
+ cat packaging/systemd/$i.service | sed -e 's@\[Service\]@[Service]\nEnvironment=KRB5CCNAME=/run/samba/krb5cc_samba@g' >tmp$i.service
install -m 0644 tmp$i.service %{buildroot}%{_unitdir}/$i.service
done
%if %with_clustering_support
@@ -1481,1 +1481,2 @@
%{_mandir}/man8/samba-tool.8*
+%{_unitdir}/samba.service
%else # with_dc
@@ -1738,6 +1743,7 @@
%{_libdir}/samba/libHDB-SAMBA4-samba4.so
%{_libdir}/samba/libasn1-samba4.so.8
%{_libdir}/samba/libasn1-samba4.so.8.0.0
+#%{_libdir}/samba/libdfs_server_ad.so
%{_libdir}/samba/libgssapi-samba4.so.2
%{_libdir}/samba/libgssapi-samba4.so.2.0.0
%{_libdir}/samba/libhcrypto-samba4.so.5
here is how to upgrade the rpm from uptream.
cd /smeserver/rpms/samba/sme10/
make clean
make prep
cvs update -dPA
# here check the last revision of patch and reuse it
cvs status samba.spec.patch
# check the last version of samba and get its link in the following command
wget https://vault.centos.org/...../samba-4.6.2-12.el7_4.src.rpm
common/cvs-import.sh -b sme10 samba-4.6.2-12.el7_4.src.rpm -m "upgrade to samba-4.6.2-12"
cvs update -dPA
make clean
make prep
#update here the correct version of patch, ie 1.3 to something else
cvs update -p -r 1.3 samba.spec.patch > samba.spec.patch
cvs add samba.spec.patch
patch <samba.spec.patch
if patch fails, you need to update the patch, see bug https://bugs.contribs.org/show_bug.cgi?id=10429 as reference.
Then edit the spec file to add changelog lines and update release, commit and build. Ideally in the changelog paste the elements from previous spec files of SME related work, so we keep track
Also to note there are some upstream related package that should not be updated (ie, not pushed to smeupdates or smeos) without rebuilding samba locally first :
- libtevent
- python-tevent
also the following are exclude as they are build locally with samba srpm
- samba,samba-client,samba-client-libs,samba-common,samba-common-libs,samba-common-tools,samba-libs,samba*,libsmbclient,libwbclient
Anaconda packages.
This is a special case. We always rebuild our installed to include our special way. Here is the list of patch for SME9
Patch10001: 0001-anaconda.id.displayMode.patch Patch10002: 0002-RemovePartitionTypeDialog.patch Patch10003: 0003-SMEServerBranding.patch Patch10004: 0004-UTC.patch Patch10005: 0005-DegradedRAID1.patch Patch10006: 0006-InstallerHDWarning.patch Patch10007: 0007-VolGroup.patch Patch10011: 0011-CheckArch.patch Patch10012: 0012-CheckArch2.patch Patch10013: 0013-Limit-languages.patch Patch10014: 0014-Make-boot-loader-use-SME-labels.patch Patch10015: 0015-Determine-upgradability-of-SME-server.patch Patch10016: 0016-Automatically-upgrade-bootloader-if-necessary.patch Patch10017: 0017-Headless-install.patch Patch10018: 0018-RaidN.patch Patch10019: 0019-UpgradeText-whitespace.patch Patch10020: 0020-PreventTooOldUpgrades.patch Patch10021: 0021-RaidN-v2.patch Patch10022: 0022-InstallerLanguages.patch Patch10023: 0023-remove_empty_line_in_lang_table.patch
then in %prep after %setup and centos patches add:
%patch10001 -p1 %patch10002 -p1 %patch10003 -p1 %patch10004 -p1 %patch10005 -p1 %patch10006 -p1 %patch10007 -p1 %patch10011 -p1 %patch10012 -p1 %patch10013 -p1 %patch10014 -p1 %patch10015 -p1 %patch10016 -p1 %patch10017 -p1 %patch10018 -p1 %patch10019 -p1 %patch10020 -p1 %patch10021 -p1 %patch10022 -p1 %patch10023 -p1
then update the changelog, the release version is the same as upstream and we add trailing release number :
Release: 1%{?dist}.1
If you are lucky the previous patches will still work, otherwise you will need to manually modify and create new patches.
Contribs.
Not part of the updatesteam maintenance cycle. Verification and release are managed by individual maintainers, however contribs packages are created in smetest, then moved direct to smecontribs - they do not appear in smeupdates-testing.
Other packages to pay attention
GeoIP on SME 8 and SME9: the package present in smetest should not be pushed (GeoIP-1.6.5-2), as it creates some conflicts. Hence, we keep GeoIP-1.4.8-1.
BUILD SYSTEM
The build system generates automatically all categories of packages by means of a script developed by Shad Lords. The script looks for all packages in higher level repos (contribs, updates, os, extras, etc) and checks for newer version in centos, epel, atrpms, rpmforge, buildsys. If it finds a newer version then it will automatically move it to the smetest repo. This is where developers/contributors should look for possible updates. No checking has been done at this point and quiet often the packages pulled automatically aren't fully compatible.
In most instances, the script will see newly built packages and put them in smetest automatically. All transaction are the subject of a repository update email (from releases@contribs.org) to the updatesteam list. For example, as soon as a new tzdata package from centos becomes available for sme8, the new package is dropped into smetest.
An automated email is also generated:
Subject:[updatesteam] Repository Update Date: Mon, 26 Nov 2012 10:47:31 -0700 From: releases@contribs.org (Releases) To: updatesteam@lists.contribs.org tzdata (sme8) copy from centos to smetest (tzdata-2012i-2.el5.i386.rpm) copy from centos to smetest (tzdata-2012i-2.el5.src.rpm) copy from centos to smetest (tzdata-2012i-2.el5.x86_64.rpm) rebuild repo (sme8) rebuild smetest/i386 rebuild smetest/x86_64 _______________________________________________ SME Server maintenance team internal discussion To unsubscribe, e-mail updatesteam-unsubscribe@lists.contribs.org Searchable archive at http://lists.contribs.org/mailman/private/updatesteam/
However, if the build system has never seen a package before, the package will end up in smedev instead of smetest. Smedev is where all first time built packages end up. It is also where all the extra packages that come out of a build end up if we don't use them. An example is php. We don't use all the packages that come out of the php srpm. The ones we don't use end up in smedev while the rest that we do would progress from smetest -> smeupdates-testing -> smeupdates -> smeos. Smetest and smedev are basically on the same level but are separate so the developers don't have to sift through all the unwanted/unneeded packages. If a package becomes wanted or needed then move it from smedev to smetest.
All packages we know about are checked. If a developer adds a new dependency that we haven't seen before, then the developer will need to manually scp the file into smetest on first use for it to be processed and added to the distro. After that the script will check for newer/updated versions and drop them automatically in smetest whenever a new version is available. To build a package (the buildsys will choose itself where it goes) simply send the build command from your environement (see the wiki page for package build SME Server:Documentation:Developers Manual:Section3).
Each repository has a max number of versions and releases that it can contain. Most by default are set to 1 for both values. The exceptions are smetest that has versions = 2 and release = 3 and smeupdates-testing which has version = 1 and release = 3. It means that the test repo can have two different version streams (devel and prod) as well as several releases on each version.
When the developer is satisfied that a package won't break the system and wants wider testing without actually releasing things, the package should be moved manually into smeupdates-testing. This is a semi-stable testing area that shouldn't break a system but hasn't been verified yet. This repo is also included in Shad's staging area (stage) for the purpose of building the next ISO. All verifications should be done on packages in smeupdates-testing. Once the package is verified, and agreement reached on the Go/NoGo for releasing, the package should be copied manually to smeupdates and released.
MOVING PACKAGES FROM ONE REPO TO ANOTHER
The update cycle involves moving packages manually from:
- smedev to smetest (infrequent)
- smetest to smeupdates-testing
- smeupdates-testing to smeudates
Packages can be moved (mv) or copied (cp). However in Build, the deletion of files from folders is prohibited, thus (mv) will lead to a (cp). For all intent and purposes, only the copy command (cp) is required. These operations are supervised by an automated update script. This script runs every two hours and checks all repositories and tries to do the right thing as far as moving packages around and cleaning up where packages came from. The automated scripts will take care of ensuring all the right actions (eg multiple RPMs, SRPM, removal of older versions) are taken for a particular package, look at the repository update emails (emails from releases@contribs.org) to the updatesteam list to see how it flows. For multiple RPMs (eg clamav) it is only needed to move one RPM, and then the whole family of RPMs and the SRPM moves. Also the scripts ensure that there is always at least one SRPM associated with a family of RPMs (noting that multiple packages can be built from one rpm spec file/tarball/src.rpm).
So how to move packages? As an example, lets take a clamav set of three RPMS available for version 7. As soon as available, the build system automated script will drop them in smetest.
Lets move them to smeupdates-testing for verification and future release:
- SSH into shell.contribs.org
- Copy the package:
$ cd /teams/smeserver/7 $ cp test/clamav-0.97.6-1.el4.rf.i386.rpm updates-testing/
Within two hours max, the automated scripts will move this package AND all other relevant packages, so just pick any one of the three clamav RPMs and let Shad’s magic take care of the real moving. The script will also delete older packages, as applicable. Note that Shad's staging area (stage) used to build the next ISO is also refreshed automatically.
An automated email is also generated:
Subject: [updatesteam] Repository Update Date: Tue, 20 Nov 2012 22:47:43 -0700 From: releases@contribs.org (Releases) To: updatesteam@lists.contribs.org clamav (sme7) ============= delete from smeupdates-testing (i386, clamav-0.97.5-2.el4.rf.i386.rpm) delete from stage (i386, clamav-0.97.5-2.el4.rf.i386.rpm) delete from smeupdates-testing (x86_64, clamav-0.97.5-2.el4.rf.x86_64.rpm) delete from stage (x86_64, clamav-0.97.5-2.el4.rf.x86_64.rpm) delete from smeupdates-testing (SRPMS, clamav-0.97.5-2.rf.src.rpm) delete from stage (SRPMS, clamav-0.97.5-2.rf.src.rpm) copy from smeupdates-testing to stage (clamav-0.97.6-1.el4.rf.i386.rpm) delete from smetest (i386, clamav-0.97.6-1.el4.rf.i386.rpm) move from smetest to smeupdates-testing (clamav-0.97.6-1.el4.rf.x86_64.rpm) copy from smeupdates-testing to stage (clamav-0.97.6-1.el4.rf.x86_64.rpm) move from smetest to smeupdates-testing (clamav-0.97.6-1.rf.src.rpm) copy from smeupdates-testing to stage (clamav-0.97.6-1.rf.src.rpm) delete from smeupdates-testing (i386, clamav-db-0.97.5-2.el4.rf.i386.rpm) delete from stage (i386, clamav-db-0.97.5-2.el4.rf.i386.rpm) delete from smeupdates-testing (x86_64, clamav-db-0.97.5-2.el4.rf.x86_64.rpm) delete from stage (x86_64, clamav-db-0.97.5-2.el4.rf.x86_64.rpm) move from smetest to smeupdates-testing (clamav-db-0.97.6-1.el4.rf.i386.rpm) copy from smeupdates-testing to stage (clamav-db-0.97.6-1.el4.rf.i386.rpm) move from smetest to smeupdates-testing (clamav-db-0.97.6-1.el4.rf.x86_64.rpm) copy from smeupdates-testing to stage (clamav-db-0.97.6-1.el4.rf.x86_64.rpm) delete from smeupdates-testing (i386, clamd-0.97.5-2.el4.rf.i386.rpm) delete from stage (i386, clamd-0.97.5-2.el4.rf.i386.rpm) delete from smeupdates-testing (x86_64, clamd-0.97.5-2.el4.rf.x86_64.rpm) delete from stage (x86_64, clamd-0.97.5-2.el4.rf.x86_64.rpm) move from smetest to smeupdates-testing (clamd-0.97.6-1.el4.rf.i386.rpm) copy from smeupdates-testing to stage (clamd-0.97.6-1.el4.rf.i386.rpm) move from smetest to smeupdates-testing (clamd-0.97.6-1.el4.rf.x86_64.rpm) copy from smeupdates-testing to stage (clamd-0.97.6-1.el4.rf.x86_64.rpm) rebuild repo (sme7) =================== rebuild smeupdates-testing/i386 rebuild smeupdates-testing/x86_64 rebuild smetest/i386 rebuild smetest/x86_64
VERIFICATION
A package should be verified using the standard verification template on a clean system, check SME Server:Documentation:QA:Verification for guidelines. The verification phase should also check for hard dependencies which could break an update, and check for correct changelogs, including a reference to a bug number in bugzilla.
RELEASING UPDATES
Once verification is completed, it is customary to flag an impending release and seek comments using the updatesteam@lists.contribs.org. There may also be a need to deal with dependencies.
The final step after any comments and package dependencies have been addressed consists in the actual release. To release a package, just copy it into the updates directory on the build system.
As an example:
- SSH into shell.contribs.org
- Copy the package:
$ cd /teams/smeserver/7 $ cp updates-testing/clamav-0.97.6-1.el4.rf.i386.rpm updates/
The automated script will move all related packages and delete older packages, as applicable. An automated email is also generated:
Subject:[updatesteam] Repository Update Date: Wed, 21 Nov 2012 16:47:46 -0700 From: releases@contribs.org (Releases) To: updatesteam@lists.contribs.org clamav (sme7) ============= delete from smeupdates-testing (i386, clamav-0.97.6-1.el4.rf.i386.rpm) move from smeupdates-testing to smeupdates (clamav-0.97.6-1.el4.rf.x86_64.rpm) move from smeupdates-testing to smeupdates (clamav-0.97.6-1.rf.src.rpm) delete from smeupdates-testing (i386, clamav-db-0.97.6-1.el4.rf.i386.rpm) move from smeupdates-testing to smeupdates (clamav-db-0.97.6-1.el4.rf.x86_64.rpm) delete from smeupdates-testing (i386, clamd-0.97.6-1.el4.rf.i386.rpm) move from smeupdates-testing to smeupdates (clamd-0.97.6-1.el4.rf.x86_64.rpm) rebuild repo (sme7) =================== rebuild smeupdates/i386 rebuild smeupdates/x86_64 rebuild smeupdates-testing/i386 rebuild smeupdates-testing/x86_64
The final steps after release consist of:
- a) Issuing a release note.
- b) Updating the forums, this is optional except when releasing a new version number.
- c) Closing all bugs associated with released packages.
DEALING WITH PACKAGE DEPENDENCIES.
|
This topic is incomplete and requires additional information |
There are two types of package dependencies to check before a release can take place:
- a) Hard dependencies. These dependencies are specified in the spec file, eg: requires: e-smith-lib >= 2.0.0-2. Developers put these in when it is necessary to enforce dependencies between packages. The verification phase taking place before release should pick up hard dependencies which could break an update - the installation and testing of a new rpm will show whether things break or not. To the best of my understanding, the 'Verified Package Versions' tab in Rnotes does not track hard dependencies. To be confirmed.
- b) Soft dependencies. These dependencies occur when a package fixes three bugs and one of them has not been verified. As a result, and until all bugs associated with a particular package are verified, only an older version of that package can be safely released. Also note that unresolved soft dependencies may affect other packages if one of the fixes involves multiple packages. Luckily the 'Verified Package Versions' tab in Rnotes will track this - Rnote script do what Ian Wells used to do manually and try to build a bug and package dependency graph.
As an illustration of soft deps, consider the following scenario, actual bug numbers etc are made up for this example:
e-smith-base 5.0.0-16 XXX 5.0.0-16.sme VERIFIED - some text some text some text [SME: 6262] XXX 5.0.0-15.sme RESOLVED - some text some text some text [SME: 6248] XXX 5.0.0-14.sme VERIFIED - some text some text some text [SME: 6104]
e-smith-pop3 2.0.0-2 XXX 2.0.0-2.sme VERIFIED - some text some text some text [SME: 6262] XXX 2.0.0-1.sme VERIFIED - some text some text some text [SME: 4633]
e-smith-apache 2.0.0-7 XXX 2.0.0-7.sme VERIFIED - some text some text some text [SME: 6769] XXX 2.0.0-6.sme VERIFIED - some text some text some text [SME: 4633]
There are three modified packages, only bug 6248 has not yet been verified.
- e-smith-base 5.0.0-16 cannot be released as it has a change that has not been verified.
- It may be possible to release e-smith-base 5.0.0-14
- e-smith-pop3 2.0.0-2 cannot be released as e-smith-base 5.0.0-16 which contains part of the fix for bug 6262 is not being released.
- e-smith-apache 2.0.0-7 cannot be released as part of the fix for Bug 4633 is in e-smith-pop3 2.0.0-1 which is not being released.
If you carefully check the code changes it may be possible to release e-smith-pop3 and e-smith-apache in the example above, but safest not to do so.
Note: Shad's automated script doesn't do any blocking of packages. It just tries to sort out where packages should belong based on where things exist when it starts running. Accordingly, the script does not prevent the release of packages for which package dependencies have not been resolved.
|
Last, we need to explain how dependencies setup between bugs in Bugzilla operate, and how they relate to hard/soft package dependencies if applicable. |
ISSUE RELEASE NOTES
Wait for the mirrors to sync, and then email manually the release notes. Release notes can be prepared manually, or one can access ALL draft release notes for sme7 and sme8 by just looking at http://wellsi.homeip.net/rnotes/notes/ . Check the "Bug Report" in the section about RNOTES below for details.
Example of release notes:
Subject:[updatesannounce] SME Server 7 Update: clamav-0.97.6-1 Date: Wed, 21 Nov 2012 23:24:53 -0800 From: Ian Wells <esmith@wellsi.com> To: updatesannounce@contribs.org -------------------------------------------------------------------------------- SME Server Update Notification 2012-11-21 -------------------------------------------------------------------------------- Name : clamav Product : SME 7 Version : 0.97.6 Release : 1.el4.rf URL : [1] Summary : Anti-virus software Description : Clam AntiVirus is a GPL anti-virus toolkit for UNIX. The main purpose of this software is the integration with mail servers (attachment scanning). The package provides a flexible and scalable multi-threaded daemon, a command line scanner, and a tool for automatic updating via Internet. The programs are based on a shared library distributed with the Clam AntiVirus package, which you can use with your own software. Most importantly, the virus database is kept up to date -------------------------------------------------------------------------------- Update Information: Update to ClamAV 0.97.6 -------------------------------------------------------------------------------- ChangeLog: * Sun Sep 23 2012 Dag Wieers <dag@wieers.com> - 0.97.6-1 - Updated to release 0.97.6. -------------------------------------------------------------------------------- References: [ 1 ] Bug 7160 - SME7.6 Need to release clamav version: 0.97.6 http://bugs.contribs.org/show_bug.cgi?id=7160 [ 2 ] ClamAV 0.97.6 Release Announcement http://www.gossamer-threads.com/lists/clamav/announce/55863 [ 3 ] ClamAV 0.97.6 Changelog http://github.com/vrtadmin/clamav-devel/blob/0.97/ChangeLog -------------------------------------------------------------------------------- Updated packages: clamav-0.97.6-1.el4.rf.i386.rpm clamav-db-0.97.6-1.el4.rf.i386.rpm clamd-0.97.6-1.el4.rf.i386.rpm clamav-0.97.6-1.el4.rf.src.rpm This update can be installed with the Software Installer from the Server Manager. http://wiki.contribs.org/SME_Server:Documentation:Administration_Manual:Chapter13#Software_Installer_Panel
MANAGING UPDATE AND RELEASE NOTES USING RNOTES.
Ian Wells has over the years developed a webpage intended to assist in the management of pending updates and the creation of release notes. It consists of a number of perl scripts that generate a site at http://wellsi.homeip.net/rnotes/. Whilst the information found there is not essential to any update, it is a valuable tool designed to streamline the overall process. It does an analysis of the packages in smeupdates-testing (both SME 7 & SME 8) and tries to work out what can be released, and if there are problems.
The site is subdivided into one html file per SME Server release, it updates every 8 hours.
SME 8: http://wellsi.homeip.net/rnotes/sme8/8bugreport.html
SME 7: http://wellsi.homeip.net/rnotes/bugreport.html
Each of the section above have a number of tabs, they are:
Summary
Summary of bugs for relevant version of SME on the day of query
Overview
Release Notes Overview - Work in progress - on the day of query
SRPMs
SRPMs content (note that the SRPMs list may be shorter than the RPMs list because some SRPMs generates multiple RPMs depending on the .spec file).
RPMs
RPMs content.
Bug Report
Report of packages associated with each bug. there are also links to relevant Bugs (click on the Bug number) and to an automated release notes (click on "notes"). One can also access ALL draft release notes for sme7 and sme8 by just looking at http://wellsi.homeip.net/rnotes/notes/
Status.
Status of each bug. The bugzilla product is highlighted if not matching the SME Version. [You may see entries like ‘5908 NA <No Product Found> Access Denied’ that means it is a security bug.]
Simple.
A simple list of bugs
Missing Bugs Numbers.
Changes without bug numbers - should always be empty if our devs remember to include a bug number in the changelog.
Verified Package Versions
This gives some hints to which packages can be released to smeupdates. It does not yet check the depends/blocks from bugzilla. It also only goes green when a bug and all dependents are Verified. When it cannot retrieve the status it colours them black, manual checking is advised.
Smetest Files
This tab lists packages provided as part of SME Server release which have a newer version. Only files that were on the ISO are checked and displayed - it is likely that these should be moved to smeupdates-testing. Note that Smetest may also contain other updated files related to contribs - and potentially other files. These are not displayed in RNotes and need to be sorted manually.
- Note 1: when compiling status of updates it is good to cross-reference the actual packages (RPMs) in the release queue (smetest/smeupdates-testing) with the bugzilla matrix and investigate where they don't match.
- Note 2: final release notes consist of a script generated templates with some manual additions. Ideally the only changes needed to the automatically generated release notes would be the description of the update, in the part ‘<Insert Update Text Here>’ in the template. What was wrong, what was fixed, and any additional end-user information. This is the only challenging part. It is good to put links to additional information in the References section, as an example, check the release note for clamav 0.97.6-1 above.
- Note 3: the template is only an aid, it may not be correct. You should check that:
- - The version/release match the changelog
- - The bug in references match the changelog - for security bugs, this needs to be added manually
- - The RPMs & SRPMs listed match the changelog
UNDERSTANDING PACKAGES IN SMETEST AND SMEUPDATES-TESTING
To help people learn about how to release updates for SME Server, here are some comments based on the situation with SME 7 on the 21st November 2012 - note that the situation has changed significantly since this was written.
Starting with SME 7, look at the VERIFIED link in the bug matrix.
It is currently showing two:
- 7086 need to fix mirrorlist away from ibiblio
- 7160 SME7.6 Need to release clamav version: 0.97.6
Now look at smetest & smeupdates-testing and see that there are a lot more packages. They may not need to be released, but at least they need to be understood.
Packages found in smetest: /smeserver/releases/7/smetest/SRPMS (includes contribs & upstream)
- clamav-0.97.6-1.rf.src.rpm
- e-smith-base-5.0.0-17.el4.sme.src.rpm
- fping-3.3-1.rf.src.rpm
- fping-3.4-1.rf.src.rpm
- gifsicle-1.67-1.rf.src.rpm
- nagios-plugins-1.4.16-1.rf.src.rpm
- perl-DBI-1.621-1.rfx.src.rpm
- perl-NetAddr-IP-4.061-1.rf.src.rpm
- pv-1.3.1-1.rf.src.rpm
- qpsmtpd-0.83-0.9.el4.sme.src.rpm
- rkhunter-1.3.8-2.el4.src.rpm
- smeserver-durep-1.3.0-4.el4.sme.src.rpm
- smeserver-durep-1.3.0-5.el4.sme.src.rpm
- smeserver-durep-1.3.0-6.el4.sme.src.rpm
- smeserver-htbwshaper-1.0-14.el4.sme.src.rpm
- smeserver-mailstats-0.0.3-15.el4.sme.src.rpm
- smeserver-qpsmtpd-2.0.0-8.el4.sme.src.rpm
- smeserver-qpsmtpd-2.0.0-9.el4.sme.src.rpm
- smeserver-sme7admin-1.1.1-23.el4.sme.src.rpm
- smeserver-subversion-1.4-51.el4.sme.src.rpm
- smeserver-webshare-1.0.0-10.el4.sme.src.rpm
- smeserver-zabbix-agent-0.1-51.el4.sme.src.rpm
- smeserver-zabbix-agent-0.1-52.el4.sme.src.rpm
- unrar-4.2.3-1.rf.src.rpm
Packages found in smeupdates-testing: /smeserver/releases/7/smeupdates-testing/SRPMS/
- clamav-0.97.5-2.rf.src.rpm
- e-smith-apache-2.0.0-7.el4.sme.src.rpm
- e-smith-backup-2.0.0-38.el4.sme.src.rpm
- e-smith-base-5.0.0-16.el4.sme.src.rpm
- e-smith-email-5.0.0-10.el4.sme.src.rpm
- e-smith-pop3-2.0.0-2.el4.sme.src.rpm
- qpsmtpd-0.83-0.8.el4.sme.src.rpm
All the non-contrib packages from smetest should be moved to smeupdates-testing for verification/testing. Essentially if you can find an older package in smeos then the package should be moved to smeupdates-testing.
Note that smetest has two versions of smeserver-qpsmtpd, only move the one that you want to smeupdates-testing Also note the (soft) dependencies, Bug 6141 involves e-smith-base & smeserver-qpsmtpd.
Packages to be moved from smetest into smeupdates-testing fall into two distinct categories:
- a) “SME Server Core packages”
- e-smith-base-5.0.0-17.el4.sme.src.rpm * Wed Jul 18 2012 Ian Wells <esmith@wellsi.com> 5.0.0-17.sme - Make CipherSuite secure by default [SME: 6141]
- qpsmtpd-0.83-0.9.el4.sme.src.rpm * Wed Jul 18 2012 Ian Wells <esmith@wellsi.com> 0.83-0.9.sme - Fix fatal errors when mail has no headers [SME: 6386]
- smeserver-qpsmtpd-2.0.0-9.el4.sme.src.rpm * Wed Jul 18 2012 Ian Wells <esmith@wellsi.com> 2.0.0-9.sme - Revert the 2.0.0-8 change and fix properly in e-smith-base [SME: 6141]
- b) “upstream” packages
- clamav-0.97.6-1.rf.src.rpm
- perl-DBI-1.621-1.rfx.src.rpm
- perl-NetAddr-IP-4.061-1.rf.src.rpm
- pv-1.3.1-1.rf.src.rpm
Once all the relevant packages are in smeupdates-testing some sensible discussion can be had on what testing is outstanding, and what can be released.
PACKAGES UPDATE CYCLE
The life of packages under the responsibility of the updatesteam starts in smetest. Leaving aside contribs (they are the responsibility of individual maintainers), there are three distinct scenarios:
Scenario One: SME modified packages
- Step one. Packages created in smetest should be summarely tested by the developer to ensure that it installs correctly. Particular attention should be given to accurate changelog including a reference to the bug report. The bug is then resolved FIXED, and a copy of the changelog including package name provided at time of resolution.
- Step two. The package can then be moved by the developer to smeupdates-testing. Essentially if you can find an older package in smeos then the package should be moved to smeupdates-testing. In practice, members of the updatesteam should ensure that the move has been done, and move all relevant packages on a weekly basis.
- Step three. Verification takes place, the bug is either REOPENED or resolved VERIFIED. Updatesteam to circulate request for comment about release after having ensured that all deps are satisfied. This could be scheduled in batch weekly unless urgent fix.
- Step four. The package is then moved to smeupdates for release, release notes issued and possibly announce in the Forums if applicable. This could be scheduled in batch weekly unless urgent fix.
Scenario Two: clamav packages
- Step one. Updatesteam raises a bug as soon as packages appear in smetest.
- Step two. Moved to updates-testing.
- Step three. Fast track verification and release + release notes.
Scenario Three: Upstream packages and kernel mods excluding clamav
|
This topic is incomplete and requires additional information |
- Step one. Identify relevant packages by looking at all the RPMs, Changelogs, and Bugzilla. To find the upstream RPMs compare smetest to smeos. Rnotes simplify this process.
- Step two. Relevant packages are moved to updates-testing.
- Step three. Some testing may take place. To be clarified
- Step four. If satisfied, packages are moved to updates.
- Step five. No formal release notes, but Ian usually send an email and post to the forums a list of packages, e.g:
Subject: [updatesannounce] SME Server 7 Update: Upstream packages 01 December 2012 Date: Sat, 01 Dec 2012 18:28:21 -0800 From: Ian Wells <esmith@wellsi.com> To: updatesannounce@contribs.org SME Server 7 Update: Upstream packages 01 December 2012 Here is the list of upstream packages that were pushed as updates. gifsicle-1.67-1.el4.rf.i386.rpm perl-DBI-1.621-1.el4.rfx.i386.rpm perl-NetAddr-IP-4.061-1.el4.rf.i386.rpm pv-1.3.1-1.el4.rf.i386.rpm ___________________________________________ Announcement of maintenance updates for SME Server To unsubscribe, e-mail updatesannounce-unsubscribe@lists.contribs.org Searchable archive at http://lists.contribs.org/mailman/public/updatesannounce/
Note 1:: In some instances, centos upstream packages found in smetest and released by us do not show in yum updates notifications on servers in the field. This is because centos has released these packages before we do. We have centos repositories enabled as well as our repositories. We house all the packages necessary for the distro in our repositories. If we ever wanted to disable centos repostories we could do so and still continue to have people with functioning systems and receiving updates. It doesn't hurt (most of the time) to have the centos repositories enabled on servers in the field. It just makes systems get updates to upstream packages quicker. We really should be putting anything that people are able to pull from centos into smeupdates as soon as possible. (Shad)
Note 2: Some packages found in smetest belong to smeaddons. If it isn't part of the distro then it doesn't go in smeupdates-testing. smeaddons is just like smecontribs but contains package that were necessary to aid upgrading from 7 to 8. They should be on a similar release schedule to what smecontribs would be. To be confirmed: where to move new packages associated with smeaddons? AND who is responsible for the release of smecontribs related packages (eg unrar)?
ABOUT THE MIRRORS
Ibiblio appears to be unreliable, making it difficult to track changes. This is because Ibiblio is experiencing intermittent full hard drive issues. All downstream mirrors from them thus also have issues. Historically ibiblio was the place to see if packages had propagated out of the buildsys.
We've recently moved our primary mirror to a new set of mirrors:
- mirror.canada.pialasse.com : Canada, Montréal (known as the CA Mirror)
- mirror.pialasse.com : in France, Roubaix near Lille.
The CA Mirror is the one that you should check for what should actually be on the mirrors. You can always check to see what mirrors systems should be using for updates by looking at the mirrorlist files that are generated and housed on the CA master mirror.
As an example: http://mirror.canada.pialasse.com/mirrorlist/smeupdates-8
Both mirrors can be used for rsync subject to being added to the list of allowed client (canadian server or french ones). Alternatively, mirrorservice.org has reopened their mirror with a 4 hour sync and a public rsync service ( they are based in UK and sync on the canadian mirror)
Current SME Server contribs.org Mirror Tree
this is the current tree of Mirror for SME Server, so we can see what mirrors are affected by issues or outages. For a full list of our download mirrors see our download locations
