Setting up RPM Building for SME Server

	Skill level: Developer
	Risk of inconsistencies with Koozali SME Server methodology, upgrades & functionality is high. One must be knowledgeable about how changes impact their Koozali SME Server. Significant risk of irreversible harm.

Setting up RPM Building for SME Server

http://www.openfusion.net/linux/mocking_rpms

Mock is a Fedora project that allows you to build RPM packages within a chroot environment, allowing you to build packages for other systems than the one you're running on (e.g. building CentOS 4 32-bit RPMs on a CentOS 5 64-bit host), and ensuring that all the required build dependencies are specified correctly in the RPM spec file.

Prerequisites

Installation Centos6 minimal

• It is recommended that you use the latest CentOS 6 (64bit) version as a base http://mirror.centos.org/centos/6/isos/x86_64/ this has been verified to build for SME 7, 8 & 9 and for i386 &x86_64

Just install a minimal el6 installation. You can use either the DVD, the minimal CD install, a net install with PXE, it's up to you.

• Setup network:

To start the networking

[root@buildcos9 ~]# /etc/sysconfig/network-scripts/ifup-eth eth0

• Set DHCP to install basic utilities

[root@buildcos9 ~]#  dhclient eth0

• Install utilities

[root@buildcos9 ~]# yum install nano system-config-network-tui mc man htop top iftop iptraf screen

• Finalise network setup

[root@buildcos9 ~]# nano /etc/sysconfig/network-scripts/ifcfg-eth0

and set manually:

DEVICE="eth0"
BOOTPROTO="none"
HWADDR="00:E0:4C:68:21:6D"
IPADDR=192.168.xxx.12
NETMASK=255.255.255.0
GATEWAY=192.168.xxx.3
DNS1=192.168.xxx.3 
IPV6INIT=no
NM_CONTROLLED="yes"
ONBOOT="YES"
TYPE="Ethernet"
UUID="fe6ebc0d-dbee-4bf6-a3cf-5e84c5453549"

Check and restart:

[root@buildcos9 ~]# chkconfig network on
[root@buildcos9 ~]# service network restart

• Enable SSH

SSH is present but is disabled at this stage. To enable ssh issue:

service sshd start

• Reboot system to ensure all new config are permanent

 [root@buildcos9 ~]# reboot

• Disable SELinux

It will be easier to see what's going on (and turn off selinux at kernel level, just to be sure)

[root@buildcos9 ~]# sed -i -e 's/rhgb quiet/selinux=0/g' /boot/grub/grub.conf
[root@buildcos9 ~]# sed -i -e 's/SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

• Remove selinux-policy-targeted and authconfig -they conflict with some e-smith/smeserver packages

[root@buildcos9 ~]# yum remove selinux-policy-targeted authconfig

• Apply all available updates.

yum update

• You need to have EPEL repository configured as EPELs version of mock will be used

# Enable EPEL for Centos 6
rpm -Uvh http://fr2.rpmfind.net/linux/epel/6/i386/epel-release-6-8.noarch.rpm

• You need to create a new user, for mock will not accept user root for building packages

useradd <username>
passwd <username>

The new user needs to be granted sudo rights

yum install sudo
/usr/sbin/usermod -aG wheel <username>

Edit the file /etc/sudoers:

nano /etc/sudoers

go down and find the following lines:

 ## Allows people in group wheel to run all commands
 # %wheel	ALL=(ALL)	ALL

and remove the hash in front of %wheel so it looks like...

## Allows people in group wheel to run all commands
%wheel	ALL=(ALL)	ALL

Log out as root and login as <username>

So then the installation of mock is just:

Installing Mock

Install mock, make and rpm-build and python-ctypes packages (the latter for better setarch support)

sudo yum --enablerepo=epel install mock python-ctypes glances
sudo yum install make rpm-build

Add yourself to the 'mock' group that will have now been created

sudo /usr/sbin/usermod -G mock <username>

The mock package creates an /etc/mock directory with configs for various OS versions (mostly Fedoras). Shad has provided working configurations for SME Server here http://mirror.canada.pialasse.com/contribs/slords/mock/

Copy all the .cfg files into /etc/mock including overwriting the site-defaults.cfg

• Remove dup file as root

[root@buildcos9 mock]# cd /etc/mock/

[root@buildcos9 mock]# rm site-defaults.cfg

wget http://mirror.canada.pialasse.com/contribs/slords/mock/site-defaults.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-7-i386-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-7-i386-iso.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-7-x86_64-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-7-x86_64-iso.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-8-i386-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-8-i386-iso.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-8-x86_64-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-8-x86_64-iso.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-9-i386-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-9-i386-iso.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-9-x86_64-base.cfg
wget http://mirror.canada.pialasse.com/contribs/slords/mock/smeserver-9-x86_64-iso.cfg

• Change group ownership to mock:

[root@buildcos9 mock]# chown root:mock /etc/mock/*

Testing building environment

You can then run a test by doing:

# e.g. initialise a smeserver-9-x86_64-base chroot environment
$ CONFIG=smeserver-9-x86_64-base
$ mock -r $CONFIG --init
# It might take a while to configure at the "Start: yum update"

which will setup an initial chroot environment using the given config. If that seemed to work (you weren't inundated with error messages), you can try a build: You will need an SRPM, for instance from http://mirror.canada.pialasse.com/releases/testing/9/smeupdates-testing/SRPMS/

# Rebuild the given source RPM within the chroot environment
# usage: mock -r <mock_config> --rebuild /path/to/SRPM e.g.
$ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/e-smith-runit-2.4.0-1.el6.sme.src.rpm

If the build succeeds, it drops your packages into the /var/lib/mock/$CONFIG/result directory. If it fails, you can check mock output, the *.log files above for more info, and/or rerun mock with the -v flag for more verbose messaging. If you get "warning: Could not canonicalize hostname: " it can be ignored. This is a DNS resolution error and the easy solution is to add your hostname to /etc/hosts

ls -1 /var/lib/mock/smeserver-9-x86_64/result
build.log
e-smith-runit-2.4.0-1.el6.sme.noarch.rpm
e-smith-runit-2.4.0-1.el6.sme.src.rpm
root.log
state.log

INSTALL AND CONFIG CVS

If this works, it is worth grabbing either CVS for one module, or the whole CVS tree using this : http://wiki.contribs.org/Simple_Package_Modification

• Setup CVS to use ssh by creating /etc/profile.d/smebuild.sh with the following content

[chris@buildcos9 profile.d]$ sudo touch smebuild.sh
[chris@buildcos9 profile.d]$ nano smebuild.sh
[chris@buildcos9 profile.d]$ sudo nano smebuild.sh
[chris@buildcos9 profile.d]$ cat smebuild.sh

# Developer environment
# This gets symlinked into /etc/profile.d

export CVS_RSH=ssh # tell CVS to use ssh

# DO NOT set CVSROOT

alias rm='rm -i'
alias cp='cp -i --preserve=timestamps'
alias mv='mv -i'

You have to logout and login again to the console for changes to take effects.

• Check CVS account

You need top create an account, then get Shad to includ you in sme group + give access to Buildsys. Can be done later...Of course it is not an issue if you have not a cvs access like we will see below, you can retrieve package in mode anonymous.

Normally you make builds from the working directory of the module you are working on, eg ~/rpms/e-smith-runit/sme9:

# Build the given module within the chroot environment
$ cd ~/rpms/e-smith-runit/sme9
$ make mockbuild

# If there is no sme9 directory do the following (to test ability to build only)
$ cd ~rpms/module_name
$ ln -s sme8 devel
$ cd devel
$ make mockbuild

	Note:
	Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance. IF 32 BITS SYSTEM: BUILDARCH=i386 make mockbuild IF 64 BITS SYSTEM: BUILDARCH=x86_64 make mockbuild

It will determine if this is for SME Server 9 or 8, and the package to build, based on which directory that you are in. The results are put under that directory.

ls -1 ~/rpms/e-smith-runit/sme9/e-smith-runit-2_4_0-2_el6_sme/
build.log
e-smith-runit-2.4.0-2.el6.sme.noarch.rpm
e-smith-runit-2.4.0-2.el6.sme.src.rpm
root.log
state.log

Note if there are any perl directory requirments :

$ rpm -qpl package_name.noarch.rpm |grep perl

Patching an upstream package

initscripts is an example of a package where we have to patch the upstream package.

Original author Ian Wells

Here it is a 3-step process, Analysis, Integration, buildsys update. The majority of the work is in steps 1 & 2 where there needs to be an analysis of what has changed, and then an integration stage where you need to check that the patches still work, update CVS with new and modified files etc. Finally the new source needs updating into the buildsys before the official build will work.

initscripts-8.45.44-3.el5.centos.src.rpm

Steps to update initscripts

Step 1: Analysis, by anyone

Check the contents of the SRPMs

$ rpm -qlp initscripts-8.45.44-3.el5.centos.src.rpm
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
initscripts-8.45.44.tar.bz2
initscripts-centos-branding.patch
initscripts.spec

$ rpm -qlp initscripts-8.45.42-2.1.el5.sme.src.rpm
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
initscripts-8.45.38-smepatches.patch
initscripts-8.45.42-kpartx.patch
initscripts-8.45.42.tar.bz2
initscripts-centos-branding.patch
initscripts.spec

Check what is different (do a directory compare)

• Identical

initscripts-centos-branding.patch (content is identical, although files are not)

• Modified

initscripts.spec
0002-create-tmp-.X11-unix-in-rc.sysinit.patch (patch is identical but line numbers changed)

• New

initscripts-8.45.44.tar.bz2 (New source will need to be added to buildsys)
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch (will be added to CVS)

• Obsolete

initscripts-8.45.42-kpartx.patch (can be removed from CVS)
initscripts-8.45.42.tar.bz2

• Not in upstream SRPM as this is an SME Server patch to be applied each time

initscripts-8.45.38-smepatches.patch

So we need the new initscripts-8.45.44.tar.bz2 & 0001-Fix-comments-in-sysctl.conf.ppc-862597.patch, and the updated 0002-create-tmp-.X11-unix-in-rc.sysinit.patch

Step 2: Integration : local mock build, can be done by a developer

• Remove the obsolete files

• Update the spec file.

Version
Release
Patches
Check the description, don't copy the 'Centos' wording to the SME package
Copy the latest upstream Changelog
Add/modify the changelog regarding SME patches

• Copy the new patch & source to the local directory

• Check that the patches apply cleanly (You don't want to see lines like Hunk #1 succeeded at 890 (offset 1 line).)

make prep

• Try to build it

make mockbuild

	Note:
	Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance. IF 32 BITS SYSTEM: BUILDARCH=i386 make mockbuild IF 64 BITS SYSTEM: BUILDARCH=x86_64 make mockbuild

• Once it builds cleanly on local mock, commit the changes to CVS. Note that you cannot submit the build yet as it will fail due to the new source.

make commit

Step 3: Buildsys access and permissions needed

Then add new sources to buildsys ON BUILDSYS

If the above step has already been done then can update to latest in CVS, and retrieve the source tarball.

• get date for import

$ date +%s
1382918459

• fix name/version/release and date in the import.log (I don't have a good explanation for this)

vi import.log*

• import the new source/tarball

make new-sources FILES=*bz2

• Finally commit changes to CVS and build it.

make commit tag build

Package Build Steps

check out package

cp -a sme8 sme9
rm -rf sme9/CVS
cd sme9
make prep

mv {sourcedir} {newsourcedir}

• example e-smith-base-5.2.0 to e-smith-base-5.4.0

rm -f *.patch

• apply any fixes from spec to tree, only for SME Server packages
• move lib/perl5/site_perl to share/perl5/vendor_perl

tar Jcvf {newsource}.tar.xz {newsourcedir}

vi *.spec

• remove patches and fixes applied directly to tree
• bump version and reset release to 1
• add changelog saying rolling for sme9

make mockbuild

• if this works then we can proceed

rpm -qpl {package-version}_sme/{rpmname}.rpm

• check that files look similar to the sme8 package

These following steps need developer access

They are here for reference, and transparency. They will need to be run on SME Server's build server. You do not need to do this part. (If you have CVS access but not access to the build servers please don't run any of these commands)

cd ..
cvs add sme9
cd sme9

date +%s

• get date for import

vi import.log*

• fix name/version/release and date

cvs add .cvsignore *.spec Makefile import.log sources

• be sure to add any other needed packages

make new-sources FILES=*xz

make commit tag build

Older notes

• To set up mock as used above see Building using Mock
• To checkout packages from CVS see Simplest way to develop patches to submit

• If you have problems on setting up the environment, ask on IRC or Devinfo.

Notes:

• Using the provided .cfg files I have verified smeserver-8-x86_64-base.cfg & smeserver-9-x86_64-base.cfg and managed to build e-smith-runit for SME 9
• the chroot environments are cached, but rebuilding them and checking for updates can be pretty network intensive, so you might want to consider setting up a local repository to pull from. mrepo (available from rpmforge) is pretty good for that.
• there don't seem to be any hooks in mock to allow you to sign packages you've built, so if you do want signed packages you need to sign them afterwards via a rpm --resign $RPMS.
• If you get an error message 'create archive failed cpio: Bad magic' then try 'yum update' as this appears to be a known problem on COS6.