Updates cycles

OVERVIEW

The SME Server update and maintenance cycle consists in the management of a number of repositories and package categories.

Repositories

The master directories associated with the update and maintenance cycles are located on the build system. Their content is then propagated to the mirrors.

They are:

• dev [corresponds to smedev in the mirrors]
• test [corresponds to smetest in the mirrors]
• updates-testing [corresponds to smeupdates-testing in the mirrors]
• updates [corresponds to smeupdates in the mirrors]

In addition, there is a staging area (stage) that is used to build the next ISO, you will see reference to it in some repository update emails.

Package Categories

Sme modified packages.

Changelog generated in-house and including a reference to a Bug in Bugzilla. Verification required before release.

Upstream packages

i.e. packages from centos, epel, atrpms, rpmforge, etc). These are either not SME modified packages, or are kernel mods. There are no bug reports or in-house changelog generated, summary testing only before release.

Clamav packages.

This is a special case. Whilst not build by SME, and due to problem having been experienced in the past, these packages (three of them) are attached to a single SME bug in bugzilla. There is no specific changelog. However, verification is recommended before release.

Contribs.

Not part of the updatesteam maintenance cycle. Verification and release are managed by individual maintainers. However, contribs packages are created in smetest, then moved direct to smecontribs - they do not appear in smeupdates-testing.

BUILD SYSTEM

The build system generates automatically all categories of packages by means of a script developed by Shad Lords. The script looks for all packages in higher level repos (contribs, updates, os, extras, etc) and checks for newer version in centos, epel, atrpms, rpmforge, buildsys. If it finds a newer version then it will automatically move it to the smetest repo. This is where developers/contributors should look for possible updates. No checking has been done at this point and quiet often the packages pulled automatically aren't fully compatible.

In most instances, the script will see newly built packages and put them in smetest automatically. All transaction are the subject of a repository update email (from releases@contribs.org) to the updatesteam list. For example, as soon as a new tzdata package from centos becomes available for sme8, the new package is dropped into smetest.

An automated email is also generated:

Subject:[updatesteam] Repository Update
Date: 	Mon, 26 Nov 2012 10:47:31 -0700
From: 	releases@contribs.org (Releases)
To: 	updatesteam@lists.contribs.org

tzdata (sme8)

copy from centos to smetest (tzdata-2012i-2.el5.i386.rpm)
copy from centos to smetest (tzdata-2012i-2.el5.src.rpm)
copy from centos to smetest (tzdata-2012i-2.el5.x86_64.rpm)

rebuild repo (sme8)

rebuild smetest/i386
rebuild smetest/x86_64
_______________________________________________
SME Server maintenance team internal discussion
To unsubscribe, e-mail updatesteam-unsubscribe@lists.contribs.org
Searchable archive at http://lists.contribs.org/mailman/private/updatesteam/

However, if the build system has never seen a package before, the package will end up in smedev instead of smetest. Smedev is where all first time built packages end up. It is also where all the extra packages that come out of a build end up if we don't use them. An example is php. We don't use all the packages that come out of the php srpm. The ones we don't use end up in smedev while the rest that we do would progress from smetest -> smeupdates-testing -> smeupdates -> smeos. Smetest and smedev are basically on the same level but are separate so the developers don't have to sift through all the unwanted/unneeded packages. If a package becomes wanted or needed then move it from smedev to smetest.

All packages we know about are checked. If a developer adds a new dependency that we haven't seen before, then the developer will need to manually scp the file into smetest on first use for it to be processed and added to the distro. After that the script will check for newer/updated versions and drop them automatically in smetest whenever a new version is available. To build a package (the buildsys will choose itself where it goes) simply send the build command from your environement (see the wiki page for package build).

Smetest can have multiple versions of a package, maybe the three newest. I am not aware whether they are archived. As they are built from CVS it should be possible to recreate one if really needed.

When the developer is satisfied that a package won't break the system and wants wider testing without actually releasing things, the package should be moved manually into smeupdates-testing. This is a semi-stable testing area that shouldn't break a system but hasn't been verified yet. This repo is also included in Shad's staging area (stage) for the purpose of building the next ISO. All verifications should be done on packages in smeupdates-testing. Once the package is verified, and agreement reached on the Go/NoGo for releasing, the package should be copied manually to smeupdates and released.

MOVING PACKAGES FROM ONE REPO TO ANOTHER

The update cycle involves moving packages manually from:

• smedev to smetest (infrequent)
• smetest to smeupdates-testing
• smeupdates-testing to smeudates

Packages can be moved (mv) or copied (cp). For all intent and purposes, only the copy command (cp) is required. These operations are supervised by an automated update scripts. This script runs every two hours and checks all repositories and tries to do the right thing as far as moving packages around and cleaning up where packages came from. The automated scripts will take care of ensuring all the right actions (eg multiple RPMs, SRPM, removal of older versions) are taken for a particular package, look at the repository update emails (emails from releases@contribs.org) to the updatesteam list to see how it flows. For multiple RPMs (eg clamav) it is only needed to move one RPM, and then the whole family of RPMs and the SRPM moves. Also the scripts ensure that there is always at least one SRPM associated with a family of RPMs (noting that one SRPM will generate one or more RPMs depending on the .spec file).

So how to move packages? As an example, lets take a clamav set of three RPMS available for version 7. As soon as available, the build system automated script will drop them in smetest.

Lets move them to smeupdates-testing for verification and future release:

1. SSH into contribs.org
2. Copy the package:

$ cd /teams/smeserver/7
$ cp test/clamav-0.97.6-1.el4.rf.i386.rpm updates-testing/

Within two hours max, the automated scripts will move this package AND all other relevant packages, so just pick any one of the three clamav RPMs and let Shad’s magic take care of the real moving. The script will also delete older packages, as applicable. Note that Shad's staging area (stage) used to build the next ISO is also refreshed automatically.

An automated email is also generated:

Subject: [updatesteam] Repository Update
Date: 	Tue, 20 Nov 2012 22:47:43 -0700
From: 	releases@contribs.org (Releases)
To: 	updatesteam@lists.contribs.org

clamav (sme7)
=============
delete from smeupdates-testing (i386, clamav-0.97.5-2.el4.rf.i386.rpm)
delete from stage (i386, clamav-0.97.5-2.el4.rf.i386.rpm)
delete from smeupdates-testing (x86_64, clamav-0.97.5-2.el4.rf.x86_64.rpm)
delete from stage (x86_64, clamav-0.97.5-2.el4.rf.x86_64.rpm)
delete from smeupdates-testing (SRPMS, clamav-0.97.5-2.rf.src.rpm)
delete from stage (SRPMS, clamav-0.97.5-2.rf.src.rpm)
copy from smeupdates-testing to stage (clamav-0.97.6-1.el4.rf.i386.rpm)
delete from smetest (i386, clamav-0.97.6-1.el4.rf.i386.rpm)
move from smetest to smeupdates-testing (clamav-0.97.6-1.el4.rf.x86_64.rpm)
copy from smeupdates-testing to stage (clamav-0.97.6-1.el4.rf.x86_64.rpm)
move from smetest to smeupdates-testing (clamav-0.97.6-1.rf.src.rpm)
copy from smeupdates-testing to stage (clamav-0.97.6-1.rf.src.rpm)
delete from smeupdates-testing (i386, clamav-db-0.97.5-2.el4.rf.i386.rpm)
delete from stage (i386, clamav-db-0.97.5-2.el4.rf.i386.rpm)
delete from smeupdates-testing (x86_64, clamav-db-0.97.5-2.el4.rf.x86_64.rpm)
delete from stage (x86_64, clamav-db-0.97.5-2.el4.rf.x86_64.rpm)
move from smetest to smeupdates-testing (clamav-db-0.97.6-1.el4.rf.i386.rpm)
copy from smeupdates-testing to stage (clamav-db-0.97.6-1.el4.rf.i386.rpm)
move from smetest to smeupdates-testing (clamav-db-0.97.6-1.el4.rf.x86_64.rpm)
copy from smeupdates-testing to stage (clamav-db-0.97.6-1.el4.rf.x86_64.rpm)
delete from smeupdates-testing (i386, clamd-0.97.5-2.el4.rf.i386.rpm)
delete from stage (i386, clamd-0.97.5-2.el4.rf.i386.rpm)
delete from smeupdates-testing (x86_64, clamd-0.97.5-2.el4.rf.x86_64.rpm)
delete from stage (x86_64, clamd-0.97.5-2.el4.rf.x86_64.rpm)
move from smetest to smeupdates-testing (clamd-0.97.6-1.el4.rf.i386.rpm)
copy from smeupdates-testing to stage (clamd-0.97.6-1.el4.rf.i386.rpm)
move from smetest to smeupdates-testing (clamd-0.97.6-1.el4.rf.x86_64.rpm)
copy from smeupdates-testing to stage (clamd-0.97.6-1.el4.rf.x86_64.rpm)

rebuild repo (sme7)
===================
rebuild smeupdates-testing/i386
rebuild smeupdates-testing/x86_64
rebuild smetest/i386
rebuild smetest/x86_64

VERIFICATION

Package should be verified using the standard verification template on a clean system. The verification phase should pickup hard dependencies which could break an update, and check for correct changelogs, including a reference to a bug number in bugzilla.

RELEASING UPDATES

Once verification is completed, it is customary to flag an impending release and seek comments using the updatesteam@lists.contribs.org. There may also be a need to deal with dependancies involving other bugs.

The final step after any comments and package dependencies have been addressed consists in the actual release. To release a package, just copy it into the updates directory on the build system.

As an example:

1. SSH into contribs.org
2. Copy the package:

$ cd /teams/smeserver/7
$ cp updates-testing/clamav-0.97.6-1.el4.rf.i386.rpm updates/

The automated script will move all related packages and delete older packages, as applicable. An automated email is also generated:

Subject:[updatesteam] Repository Update
Date: 	Wed, 21 Nov 2012 16:47:46 -0700
From: 	releases@contribs.org (Releases)
To: 	updatesteam@lists.contribs.org

clamav (sme7)
=============
delete from smeupdates-testing (i386, clamav-0.97.6-1.el4.rf.i386.rpm)
move from smeupdates-testing to smeupdates (clamav-0.97.6-1.el4.rf.x86_64.rpm)
move from smeupdates-testing to smeupdates (clamav-0.97.6-1.rf.src.rpm)
delete from smeupdates-testing (i386, clamav-db-0.97.6-1.el4.rf.i386.rpm)
move from smeupdates-testing to smeupdates (clamav-db-0.97.6-1.el4.rf.x86_64.rpm)
delete from smeupdates-testing (i386, clamd-0.97.6-1.el4.rf.i386.rpm)
move from smeupdates-testing to smeupdates (clamd-0.97.6-1.el4.rf.x86_64.rpm)

rebuild repo (sme7)
===================
rebuild smeupdates/i386
rebuild smeupdates/x86_64
rebuild smeupdates-testing/i386
rebuild smeupdates-testing/x86_64

The final steps after release consist of:

a) Isssuing a release note

b) Updating the forums, this is optional except when releasing a new version number.

c) Closing all bugs associated with released packages.

DEALING WITH PACKAGE DEPENDENCIES.

The verification phase taking place before release should pick up hard dependencies which could break an update. What needs to be checked is where a package fixes three bugs and one was not verified. That means that only an older version of that package can be released. Also it can affect other packages if one of the fixes involved multiple packages. Note that the 'Verified Package Versions' tab in Rnotes will track this - it does what Ian used to do normally and try to build a bug and package dependency graph.

We can distinghish between "hard" and "soft" dependencies referred to in this document as follows:

a) Hard dependencies are those from the spec file, eg: requires: e-smith-lib >= 2.0.0-2

Developers put these when it is necessary to enforce dependencies between packages.

b) Soft dependencies relates to package dependencies setup in Bugzilla.

As an illustration, consider the following scenario, actual bug numbers etc are made up for this example:

e-smith-base 5.0.0-16
XXX 5.0.0-16.sme VERIFIED
- some text some text some text [SME: 6262]
XXX 5.0.0-15.sme RESOLVED
- some text some text some text [SME: 6248]
XXX 5.0.0-14.sme VERIFIED
- some text some text some text [SME: 6104]

e-smith-pop3 2.0.0-2
XXX 2.0.0-2.sme VERIFIED
- some text some text some text [SME: 6262]
XXX 2.0.0-1.sme VERIFIED
- some text some text some text [SME: 4633]

e-smith-apache 2.0.0-7
XXX 2.0.0-7.sme VERIFIED
- some text some text some text [SME: 6769]
XXX 2.0.0-6.sme VERIFIED
- some text some text some text [SME: 4633]

There are three modified packages, only bug 6248 has not yet been verified.

• e-smith-base 5.0.0-16 cannot be released as it has a change that has not been verified.
  • It would be possible to release e-smith-base 5.0.0-14
• e-smith-pop3 2.0.0-2 cannot be released as e-smith-base 5.0.0-16 which contains part of the fix for bug 6262 is not being released.
• e-smith-apache 2.0.0-7 cannot be released as part of the fix for Bug 4633 is in e-smith-pop3 2.0.0-1 which is not being released.

If you carefully check the code changes it may be possible to release e-smith-pop3 & e-smith-apache in the example above, but safest not to.

To be clarified:
- If deps are not resolved, will Shad's script block cp for unresolved packages dependencies? 
[Answer from Ian: not sure for updates. But normally it is not hard dependencies being an issue. ]