Difference between revisions of "Setting up RPM Building for SME Server"

From SME Server
Jump to navigationJump to search
 
(127 intermediate revisions by 10 users not shown)
Line 6: Line 6:
 
http://www.openfusion.net/linux/mocking_rpms
 
http://www.openfusion.net/linux/mocking_rpms
  
−
Mock is a Fedora project that allows you to build RPM packages within a chroot environment, allowing you to build packages for other systems than the one you're running on (e.g. building CentOS 4 32-bit RPMs on a CentOS 5 64-bit host), and ensuring that all the required build dependencies are specified correctly in the RPM spec file.
+
Mock is a Fedora project that allows you to build RPM packages within a chroot environment, allowing you to build packages for other systems than the one you're running on (e.g. building CentOS 4 32-bit RPMs on a CentOS 5 64-bit host), and ensuring that all the required build dependencies are specified correctly in the RPM spec file. You can find more example  [[Rpm_build_example_using_cvs/mock|of how using Mock]]
  
−
It is recommended that you use the latest CentOS 6 (64bit) version as a base http://mirror.centos.org/centos/6/isos/x86_64/
+
==== Prerequisites ====
−
I have verified this with CentOS-6.3-x86_64-bin-DVD1.iso
+
{{warning box|If you plan to build for sme10, you must use an el7 version of centos minimal. It is also workable on a Fedora25}}
 +
====Installation CentOS6 / CentOS7 minimal====
 +
* It is recommended that you use the latest CentOS 6 (64bit) version as a base http://mirror.centos.org/centos/6/isos/x86_64/ this has been verified to build for SME 7, 8 & 9 and for i386 &x86_64
 +
Just install a minimal el6 installation. You can use either the DVD, the minimal CD install, a net install with PXE, it's up to you. Of course if you prefer you can use a system with a Xorg-server installed (eg: gnome, kde, xfce...)
 +
* Setup network:
 +
To start the networking
 +
[root@buildcos9 ~]# /etc/sysconfig/network-scripts/ifup-eth eth0
 +
* Set DHCP to install basic utilities
 +
[root@buildcos9 ~]#  dhclient eth0
 +
* Install utilities
 +
[root@buildcos9 ~]# yum install nano system-config-network-tui mc man htop top iftop iptraf screen openssh-clients mlocate
 +
* Finalise network setup
 +
[root@buildcos9 ~]# nano /etc/sysconfig/network-scripts/ifcfg-eth0
 +
and set manually:
  
−
You need to have EPEL repository configured as EPELs version of mock will be used
+
DEVICE="eth0"
 +
BOOTPROTO="none"
 +
HWADDR="00:E0:4C:68:21:6D"
 +
IPADDR=192.168.xxx.12
 +
NETMASK=255.255.255.0
 +
GATEWAY=192.168.xxx.3
 +
DNS1=192.168.xxx.3
 +
IPV6INIT=no
 +
NM_CONTROLLED="yes"
 +
ONBOOT="YES"
 +
TYPE="Ethernet"
 +
UUID="fe6ebc0d-dbee-4bf6-a3cf-5e84c5453549"
 +
 
 +
Check and restart:
 +
[root@buildcos9 ~]# chkconfig network on
 +
[root@buildcos9 ~]# service network restart
 +
 
 +
*  Enable SSH
 +
SSH is present but is disabled at this stage. To enable ssh issue:
 +
service sshd start
 +
 
 +
You may have a slow login to ssh, if needed :
 +
nano /etc/ssh/sshd_config
 +
 
 +
And change the “UseDNS yes” field to “UseDNS no”.
 +
Now change the “GSSAPIAuthentication yes” line to “GSSAPIAuthentication no”.
 +
 
 +
You have to restart the ssh service or the server
 +
 
 +
* Reboot system to ensure all new config are permanent
 +
  [root@buildcos9 ~]# reboot
 +
 
 +
*Disable SELinux SME9
 +
It will be easier to see what's going on (and turn off selinux at kernel level, just to be sure)
 +
[root@buildcos9 ~]# sed -i -e 's/rhgb quiet/selinux=0/g' /boot/grub/grub.conf
 +
[root@buildcos9 ~]# sed -i -e 's/SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config
 +
 
 +
*SME10
 +
in /etc/selinux/config set */SELINUX=disabled
 +
 
 +
* Remove selinux-policy-targeted and authconfig -they conflict with some e-smith/smeserver packages
 +
[root@buildcos9 ~]# yum remove selinux-policy-targeted authconfig
 +
 
 +
* Apply all available updates.
 +
<syntaxhighlight lang="Bash">
 +
yum update
 +
</syntaxhighlight>
 +
 
 +
* You need to have EPEL repository configured as EPELs version of mock will be used
 
<syntaxhighlight lang="Bash">
 
<syntaxhighlight lang="Bash">
 +
 
# Enable EPEL for Centos 6
 
# Enable EPEL for Centos 6
 
rpm -Uvh http://fr2.rpmfind.net/linux/epel/6/i386/epel-release-6-8.noarch.rpm
 
rpm -Uvh http://fr2.rpmfind.net/linux/epel/6/i386/epel-release-6-8.noarch.rpm
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
# Enable EPEL and sclo repo for Centos 7
 +
rpm -ivh <nowiki>https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-12.noarch.rpm</nowiki>
 +
or (works 29Dec19 - bjr)
 +
 +
yum install -y epel-release
 +
 +
rpm -ivh <nowiki>http://mirror.centos.org/altarch/7/extras/aarch64/Packages/centos-release-scl-rh-2-3.el7.centos.noarch.rpm</nowiki>
 +
 +
yum install centos-release-scl-rh
 +
 +
* You need to create a new user, for mock will not accept user root for building packages
 +
<div class="mw-collapsible mw-collapsed" data-expandtext="More on creating a sudo user" data-collapsetext="Collapse">
 +
 +
<syntaxhighlight lang="Bash">
 +
useradd <username>
 +
passwd <username>
 +
</syntaxhighlight>
 +
The new user needs to be granted sudo rights
 +
<syntaxhighlight lang="Bash">
 +
yum install sudo
 +
/usr/sbin/usermod -aG wheel <username>
 +
</syntaxhighlight>
 +
 +
Edit the file /etc/sudoers:
 +
<syntaxhighlight lang="Bash">
 +
nano /etc/sudoers
 +
</syntaxhighlight>
 +
go down and find the following lines:
 +
 +
<syntaxhighlight lang="Bash">
 +
## Allows people in group wheel to run all commands
 +
# %wheel ALL=(ALL) ALL
 +
</syntaxhighlight>
 +
 +
and remove the hash in front of %wheel so it looks like...
 +
<syntaxhighlight lang="Bash">
 +
## Allows people in group wheel to run all commands
 +
%wheel ALL=(ALL) ALL
 +
</syntaxhighlight>
 +
 +
Log out as root and login as <username>
 +
</div>
 +
  
 
So then the installation of mock is just:
 
So then the installation of mock is just:
  
 
==== Installing Mock ====
 
==== Installing Mock ====
−
# Install mock and python-ctypes packages (the latter for better setarch support)
+
{{note box|Mock creates chroots and builds packages in them. Its only task is to reliably populate a chroot and attempt to build a package in that chroot. Mock is currently being used for all Fedora builds. Mock takes a srpm and builds it in a chroot. This ensures that your [http://fedoraproject.org/wiki/Packaging:Guidelines#BuildRequires_2 BuildRequires] lines are correct, that there are no missing dependencies, and that it builds cleanly. The [http://fedoraproject.org/wiki/Projects/Mock Mock project] page has more information on Mock.}}
−
$ sudo yum --enablerepo=epel install mock python-ctypes
+
 
 +
Install mock, make and rpm-build and python-ctypes packages (the latter for better setarch support)
 +
<syntaxhighlight lang="Bash">
 +
sudo yum --enablerepo=epel install mock python-ctypes glances
 +
sudo yum install make rpm-build rpm-sign wget
 +
</syntaxhighlight>
 +
Add yourself to the 'mock' group that will have now been created
 +
<syntaxhighlight lang="Bash">
 +
sudo /usr/sbin/usermod -G mock <username>
 +
</syntaxhighlight>
  
−
# Add yourself to the 'mock' group that will have now been created
+
The mock package creates an /etc/mock directory with configs for various OS versions (mostly Fedoras). dani has provided working configurations for SME Server here http://mirror.contribs.org/contribs/dani/mock/
−
$ sudo usermod -G mock [youraccount]
 
−
The mock package creates an /etc/mock directory with configs for various OS versions (mostly Fedoras). The first thing you want to tweak there is the site-defaults.cfg file which sets up various defaults for all your builds. Mine now looks like this:
 
  
−
# /etc/mock/site-defaults.cfg
+
Copy all the .cfg files into /etc/mock including overwriting the site-defaults.cfg
 +
* Remove dup file as root
 +
[root@buildcos9 mock]# cd /etc/mock/
  
−
  # Set this to true if you've installed python-ctypes
+
  [root@buildcos9 mock]# rm site-defaults.cfg
−
config_opts['internal_setarch'] = True
 
  
−
  # Turn off ccache since it was causing errors I haven't bothered debugging
+
  wget http://mirror.contribs.org/contribs/dani/mock/site-defaults.cfg
−
  config_opts['plugin_conf']['ccache_enable'] = False
+
  wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-i386-base.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-i386-iso.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-x86_64-base.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-x86_64-iso.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-i386-base.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-i386-iso.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-x86_64-base.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-x86_64-iso.cfg
 +
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-10-x86_64-base.cfg
  
−
# (Optional) Fake the build hostname to report
+
* Change group ownership to mock:
−
  config_opts['use_host_resolv'] = False
+
  [root@buildcos9 mock]# chown root:mock /etc/mock/*
−
config_opts['files']['etc/hosts'] = """
 
−
127.0.0.1 buildbox.openfusion.com.au nox.openfusion.com.au localhost
 
−
"""
 
−
config_opts['files']['etc/resolv.conf'] = """
 
−
nameserver 127.0.0.1
 
−
"""
 
  
−
Alternatively Shad has provided configuration for SME Server here http://mirror.canada.pialasse.com/contribs/slords/mock/ the links to the repositories need to be updated to a relevant (local) mirror
+
* workaround (as of 11Oct2016 - this does not seem necessary - the required line is already in the cfg files - bjr)
 +
if the tag 'sme' is not added  to the built package see [[bugzilla:8636]] and add <br />
  
−
==== Setting up building enviroment ====
+
to smeserver-8-i386-base.cfg & smeserver-8-x86_64-base.cfg
−
# Setup various rpm macros to use
+
  config_opts['macros']['%dist'] = ".el5.sme"
−
  config_opts['macros']['%packager'] = 'Gavin Carr <gavin@openfusion.com.au>'
+
to smeserver-9-i386-base.cfg & smeserver-9-x86_64-base.cfg
−
  config_opts['macros']['%debug_package'] = '%{nil}'
+
  config_opts['macros']['%dist'] = ".el6.sme"
−
You can use the epel-5-{i386,x86_64}.cfg configs as-is if you like; I copied them to centos-5-{i386,x86_64}.cfg versions and removed the epel 'extras', 'testing', and 'local' repositories from the yum.conf section, since I typically want to build using only 'core' and 'update' packages.
+
 
 +
* Script to create build directories
 +
#!/bin/sh
 +
mkdir ~/rpmbuild
 +
for i in BUILD BUILDROOT RPMS SOURCES SPECS SRPMS
 +
do
 +
  mkdir ~/rpmbuild/$i
 +
done
  
 
==== Testing building environment ====
 
==== Testing building environment ====
 +
 
You can then run a test by doing:
 
You can then run a test by doing:
  
−
  # e.g. initialise a centos-5-i386 chroot environment
+
  # e.g. initialise a smeserver-9-x86_64-base chroot environment
−
  $ CONFIG=centos-5-i386
+
  [chris@build-system SRPMS] $ CONFIG=smeserver-9-x86_64-base
−
  $ mock -r $CONFIG --init
+
  [chris@build-system SRPMS] $ mock -r $CONFIG --init
−
which will setup an initial chroot environment using the given config. If that seemed to work (you weren't inundated with error messages), you can try a build:
+
 
 +
It might take a while to configure at the "Start: yum update"
 +
 
 +
which will setup an initial chroot environment using the given config. If that seemed to work (you weren't inundated with error messages), you can try a build
  
 +
=====Perform a Build for sme9 64 bits from an SRPM=====
 +
You will need an SRPM, for instance from http://mirror.contribs.org/releases/testing/9/smeupdates-testing/SRPMS/
 +
you can use '''wget''' to download it in your terminal
 
  # Rebuild the given source RPM within the chroot environment
 
  # Rebuild the given source RPM within the chroot environment
 
  # usage: mock -r <mock_config> --rebuild /path/to/SRPM e.g.
 
  # usage: mock -r <mock_config> --rebuild /path/to/SRPM e.g.
−
$ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/clix-0.3.4-1.of.src.rpm
+
  [chris@build-system SRPMS]$ CONFIG=smeserver-9-x86_64-base
−
If the build succeeds, it drops your packages into the /var/lib/mock/$CONFIG/result directory:
+
  [chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/e-smith-runit-2.4.0-1.el6.sme.src.rpm
 +
 
 +
If the build succeeds, it drops your packages into the /var/lib/mock/$CONFIG/result directory.
 +
If it fails, you can check mock output, the *.log files above for more info, and/or rerun mock with the -v flag for more verbose messaging.
 +
If you get "warning: Could not canonicalize hostname: " it can be ignored. This is a DNS resolution error and the easy solution is to add your hostname to /etc/hosts
 +
 
 +
ls -1 /var/lib/mock/smeserver-9-x86_64/result
 +
build.log
 +
e-smith-runit-2.4.0-1.el6.sme.noarch.rpm
 +
e-smith-runit-2.4.0-1.el6.sme.src.rpm
 +
root.log
 +
state.log
 +
 
 +
=====Perform a Build for sme9 32 bits from an SRPM=====
 +
[chris@build-system SRPMS]$ CONFIG=smeserver-9-i386-base
 +
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/e-smith-runit-2.4.0-1.el6.sme.src.rpm
 +
 
 +
=====Perform a Build for sme8 64 bits from an SRPM=====
 +
[chris@build-system SRPMS]$ wget http://mirror.contribs.org/releases/8.0/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm
 +
[chris@build-system SRPMS]$ CONFIG=smeserver-8-x86_64-base
 +
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm
 +
 
 +
see the built rpm in the relevant folder
 +
 
 +
ls -1 /var/lib/mock/smeserver-8-x86_64/result
 +
 
 +
=====Perform a Build for sme8 32 bits from an SRPM=====
 +
[chris@build-system SRPMS]$ CONFIG=smeserver-8-i386-base
 +
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm
 +
 
 +
====Install and config CVS====
 +
{{note box|SME Server uses CVS for maintaining code and packages. SME Server core and contribs code is stored in CVS, this is a system to maintain version history and change management for program code.
 +
There are two repositories hosting code for SME Server:
 +
 
 +
    SME Server - holding the source code for the SME Server core
 +
    SME Contribs - holding the source code for contribs (add-ons) for SME Server
 +
 
 +
Both repositories are organized by the package name under which they appear in the software installer in the server-manager (or yum).  please see [[CVS|CVS page]] for more specific information}}
 +
 
 +
If this works, it is worth grabbing either CVS for one module, or the whole CVS tree using this : http://wiki.contribs.org/Simple_Package_Modification
 +
* Setup CVS to use ssh by creating /etc/profile.d/smebuild.sh with the following content
 +
 
 +
[chris@buildcos9 profile.d]$ sudo nano smebuild.sh
 +
 
 +
Paste this:
 +
 
 +
# Developer environment
 +
# This gets symlinked into /etc/profile.d
 +
 +
export CVS_RSH=ssh # tell CVS to use ssh
 +
 +
# DO NOT set CVSROOT
 +
 +
alias rm='rm -i'
 +
alias cp='cp -i --preserve=timestamps'
 +
alias mv='mv -i'
 +
 
 +
You have to logout and login again to the console for changes to take effects.
 +
 
 +
* Check CVS account
 +
You need top create an account on shell.koozali.org, then get an admin to include you in sme group + give access to Buildsys. Can be done later...Of course it is not an issue if you have not a cvs access like we will see below, you can retrieve package in mode anonymous.
 +
 
 +
====KOOZALI CVS access====
 +
This chapter is used only by developers who aim to be granted in write access to the source code on shell.koozali.org, you don't need it if you want to only make patch or modify code in your build environment. You can see [[Package_Modification]] for more information.
 +
* SME Server code is stored in the CVS on  shell.koozali.org. To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org. With this account the development team can give you access to the CVS repository.
 +
* After you have had your account created you can ask the development team to give you developer access to smecontribs. Create a bug in the Bug Tracker as usual.
 +
 
 +
* If local username is different to koozali.org username edit ~/.ssh/config:
 +
 
 +
Host  shell.koozali.org
 +
User  koozaliusername ''(without @shell.koozali.org)''
 +
Host  shell.koozali.org
 +
User  koozaliusername ''(without @shell.koozali.org)''
 +
 
 +
* Don't forget to set the proper privileges on the file
 +
 +
chmod 600 ~/.ssh/config
 +
 
 +
the port 222 will need to be defined in ~/.ssh/config
 +
host shell.koozali.org
 +
port 222
 +
 
 +
you can also define there the rsa key you want to use...
 +
 
 +
If you have a number of different keys ssh/cvs may try and use the wrong one.
 +
 
 +
You can add the following to your ~/.ssh/config file to specify the correct key:
 +
 
 +
Host  shell.koozali.org
 +
User  koozaliusername
 +
IdentityFile ~/.ssh/my.private.key
 +
 
 +
====Using CVS - Getting source Code====
 +
We need to create directory in /home/chris for example this how you can do
 +
 
 +
exchange  - used to move files around with wget
 +
work - used for CVS jobs
 +
rpmbuild - used to build with Mock
 +
rpms - used for CVS tree download
 +
 
 +
[chris@buildcos9 work]$ cd /home/chris
 +
[chris@buildcos9 work]$  mkdir {exchange,work,rpmbuild,rpms}
 +
'''CVS is used for maintaining code and packages.'''<br />
 +
 
 +
We first need to download (with CVS this is called retrieving) the code for the package to be modified from the SME Server CVS repository at Koozali.org (we can also retrieve the CVS for the smecontribs tree).
 +
Two cases with CVS:
 +
 
 +
To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org.  If you do not have an account at CVS, retrieving is performed as an anonymous user and changes cannot be saved.
 +
 
 +
===== Case 1 - Registered Users =====
 +
- To retrieve a package or a full tree as a registered user, cd into a work directory and issue following commands:
 +
* for smeserver
 +
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P packagename [individual package]
 +
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P rpms [full tree]
 +
env CVS_RSH=ssh cvs -d:ext:YOURUSERNAME@shell.koozali.org:/cvs/smeserver checkout CVSROOT rpms common
 +
 
 +
* for smecontribs
 +
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P packagename [individual package]
 +
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P rpms [full tree]
 +
env CVS_RSH=ssh cvs -d:ext:YOURUSERNAME@shell.koozali.org:/cvs/smecontribs checkout CVSROOT rpms common
 +
 
 +
===== Case 2 - Anonymous users =====
 +
- To retrieve a package or a full tree as an anonymous user, cd into a work directory and issue following commands:
 +
* for smeserver
 +
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smeserver checkout -P packagename [individual package]
 +
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smeserver co -P rpms [full tree]
 +
 
 +
* for smecontribs
 +
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smecontribs checkout -P packagename [individual package]
 +
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smecontribs co -P rpms [full tree]
 +
 
 +
Note: If at a later stage you obtain a valid account with CVS, you will need to edit ALL packages /CVS/Root in order to save your work in CVS. As an example, for package "packagename", CVS/Root will need to be edited to be exactly as shown below:
 +
[chris@buildcos9 work]$ cat packagename/CVS/Root
 +
:ext:shell.koozali.org:/cvs/smeserver
 +
 
 +
ok - CVS/Root uses 'ext' which should be SSH access
 +
the port 222 will need to be defined in ~/.ssh/config
 +
host shell.koozali.org
 +
port 222
 +
you can also define there the rsa key you want to use...
 +
 
 +
==== Package modification ====
 +
* Prepare work areas, make some directories:
 +
[chris@buildcos9 ~]$ pwd
 +
/home/chris
 +
 
 +
* [chris@buildcos9 ~]$ ls
 +
exchange  rpmbuild  rpms  test  work
 +
 
 +
* Import cvs in your workspace
 +
[chris@buildcos9 ~]$ cd work/
 +
[chris@buildcos9 ~]$ mkdir smeserver
 +
[chris@buildcos9 ~]$ cd smeserver
 +
[chris@buildcos9 work]$ cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P packagename
 +
 
 +
or for smecontribs
 +
[chris@buildcos9 ~]$ cd work/
 +
[chris@buildcos9 ~]$ mkdir smecontribs
 +
[chris@buildcos9 ~]$ cd smecontribs
 +
[chris@buildcos9 work]$ cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P packagename
 +
 
 +
* To refresh run the following from the rpms directory, or any lower directory with a CVS dir.
 +
[chris@buildcos9 ~] cd packagename/
 +
[chris@buildcos9 packagename]$ cvs update -dPA
 +
 
 +
* Change to work directory, in this example, we are working on sme9:
 +
[chris@buildcos9 packagename] cd sme9
 +
 
 +
* To prepare a tree
 +
[chris@buildcos9 sme9]$  cvs update -dPA [if not done previously]
 +
[chris@buildcos9 sme9]$  make clean  [ignore errors about tar.xz:file missing]   
 +
[chris@buildcos9 sme9]$  make prep
 +
 
 +
* Make a patch
 +
Our patch will be a done with the 'diff' tool which compares the edited files against the original files.
 +
 
 +
* We need to to save a copy of the original files:
 +
cp -R packagename-1.2.3 packagename-1.2.3.old
 +
 
 +
* Switch to the tree and make required changes/modifications.
 +
 
 +
* Create a patch using a sensible name, eg "packagename-1.2.3-migrate_to_UNC_syntax.patch":
 +
[chris@buildcos9 sme9]$ diff -urN packagename-1.2.3.old packagename-1.2.3 > packagename-1.2.3-your_changes-description.patch
 +
 
 +
* Update the spec file
 +
[chris@buildcos9 sme9]$ nano packagename.spec
 +
 
 +
* Increase the release version at the beginning of the spec file by 1:
 +
%define release 4
 +
 
 +
* Add the patch to the existing patchlist section not forgetting to increase the patch number by 1:
 +
Patch3: packagename-1.2.3-my_changes_description
 +
 
 +
* Add patch version to the %setup section not forgetting to increase number by 1:
 +
%patch3 -p1
 +
 
 +
* Update the changelog section, including the release version and bug number:
 +
%changelog
 +
* Mon Apr 01 2013 name <email> 1.2.3-4.sme
 +
- Summary of changes [SME: xxxx]
 +
 
 +
NOTE: The formatting of date information is critical, any errors will prevent successful commit to CVS.
 +
In doubts, check previous entries in the changelog.
 +
 
 +
NOTE: There must be a bug reference for every change, it must be in the format [SME: xxxx]
  
−
  $ ls -1 /var/lib/mock/$CONFIG/result
+
* Check changes against CVS
 +
  [chris@buildcos9 sme9]$ cvs diff -u packagename.spec
 +
 
 +
* Build the test RPM
 +
[chris@buildcos9 sme9]$ pwd
 +
/home/chris/work/packagename/sme9
 +
 
 +
[chris@buildcos9 sme9]$ make mockbuild
 +
 
 +
{{Note box|msg=Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance.
 +
* Default
 +
make mockbuild
 +
* IF 32 BITS SYSTEM:
 +
BUILDARCH=i386 make mockbuild
 +
* IF 64 BITS SYSTEM:
 +
BUILDARCH=x86_64 make mockbuild}}
 +
 
 +
===== Basic Testing of Package: =====
 +
Check for built packages:
 +
[chris@buildcos9 sme9]$ ls -la packagename-1_2_3-4_el6_sme
 +
 
 +
- Send new package (and patch if required) to workstation for testing/processing, as an example:
 +
[chris@buildcos9 packagename-1_2_3-4_el6_sme]$ scp -P 22 packagename-1_2_3-4_el6_sme.noarch.rpm chris\@192.168.0.1:/home/chris/WIP
 +
 
 +
- Move package to an ibay on test server
 +
 
 +
- Install package (yum localinstall or rpm -Uvh, for SME9 you can also use yum update packagename-1_2_3-4_el6_sme.noarch.rpm)
 +
 
 +
- Ensure nothing is broken.
 +
 
 +
 
 +
======Notes on mock/cvs======
 +
Normally you make builds from the working directory of the module you are working on, eg ~/rpms/e-smith-runit/sme9:
 +
# Build the given module within the chroot environment
 +
$ cd ~/rpms/e-smith-runit/sme9
 +
$ make mockbuild
 +
 
 +
# If there is no sme9 directory do the following (to test ability to build only)
 +
$ cd ~rpms/module_name
 +
$ ln -s sme8 devel
 +
$ cd devel
 +
$ make mockbuild
 +
 
 +
It will determine if this is for SME Server 9 or 8, and the package to build, based on which directory that you are in. The results are put under that directory.
 +
 
 +
ls -1 ~/rpms/e-smith-runit/sme9/e-smith-runit-2_4_0-2_el6_sme/
 
  build.log
 
  build.log
−
  clix-0.3.4-1.of.noarch.rpm
+
  e-smith-runit-2.4.0-2.el6.sme.noarch.rpm
−
  clix-0.3.4-1.of.src.rpm
+
  e-smith-runit-2.4.0-2.el6.sme.src.rpm
 
  root.log
 
  root.log
 
  state.log
 
  state.log
−
If it fails, you can check mock output, the *.log files above for more info, and/or rerun mock with the -v flag for more verbose messaging.
+
 
 +
Note if there are any perl directory requirments :
 +
 
 +
$ rpm -qpl package_name.noarch.rpm |grep perl
 +
 
 +
===== Finalising in CVS and buildsys =====
 +
* Add the patch file to cvs:
 +
[chris@buildcos9 sme9]$ cvs add packagename-1.2.3-your_changes-description.patch
 +
cvs add: scheduling file `packagename-1.2.3-your_changes-description' for addition
 +
cvs add: use 'cvs commit' to add this file permanently
 +
 
 +
NOTE: ignore this suggestion! Just proceed to next stage or you will end up in all kinds of problems....
 +
 
 +
* Build package on Shad's system:
 +
[chris@buildcos9 sme9]$ make commit tag build
 +
 
 +
Note: this can also be done in stages:
 +
make commit
 +
make tag
 +
make build
 +
 
 +
* Check for Build:
 +
http://buildsys.contribs.org/plague/
 +
 
 +
* Check for emails received from buildsys@contribs.org, you should get one from CVS and two from the Buildsystem.
 +
 
 +
* If (and only if) build has been successful, open relevant ticket in Bugzilla and notify of build:
 +
 
 +
Fixed in packagename:
 +
%changelog
 +
* Mon Apr 01 2013 name <email> 1.2.3-4.sme
 +
- Summary of changes [SME: xxxx]
 +
 
 +
You may also wish to provide some tips about verification.
 +
 
 +
== Patching an upstream package ==
 +
 
 +
initscripts is an example of a package where we have to patch the upstream package. Note that this is not the best approach, new method to be documented by Ian soon.
 +
 
 +
Original author [http://lists.contribs.org/pipermail/devinfo/2013-October/011977.html Ian Wells]
 +
 
 +
Here it is a 3-step process, Analysis, Integration, buildsys update. The majority of the work is in steps 1 & 2 where there needs to be an analysis of what has changed, and then an integration stage where you need to check that the patches still work, update CVS with new and modified files etc. Finally the new source needs updating into the buildsys before the official build will work.
 +
 
 +
initscripts-8.45.44-3.el5.centos.src.rpm
 +
 
 +
Steps to update initscripts
 +
 
 +
===Step 1: Analysis, by anyone===
 +
 
 +
Check the contents of the SRPMs
 +
 
 +
$ rpm -qlp initscripts-8.45.44-3.el5.centos.src.rpm
 +
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch
 +
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
 +
initscripts-8.45.44.tar.bz2
 +
initscripts-centos-branding.patch
 +
initscripts.spec
 +
 
 +
$ rpm -qlp initscripts-8.45.42-2.1.el5.sme.src.rpm
 +
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
 +
initscripts-8.45.38-smepatches.patch
 +
initscripts-8.45.42-kpartx.patch
 +
initscripts-8.45.42.tar.bz2
 +
initscripts-centos-branding.patch
 +
initscripts.spec
 +
 
 +
 
 +
Check what is different (do a directory compare)
 +
 
 +
 
 +
* Identical
 +
initscripts-centos-branding.patch (content is identical, although files are not)
 +
 
 +
* Modified
 +
initscripts.spec
 +
0002-create-tmp-.X11-unix-in-rc.sysinit.patch (patch is identical but line numbers changed)
 +
 
 +
* New
 +
initscripts-8.45.44.tar.bz2 (New source will need to be added to buildsys)
 +
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch (will be added to CVS)
 +
 
 +
* Obsolete
 +
initscripts-8.45.42-kpartx.patch (can be removed from CVS)
 +
initscripts-8.45.42.tar.bz2
 +
 
 +
* Not in upstream SRPM as this is an SME Server patch to be applied each time
 +
initscripts-8.45.38-smepatches.patch
 +
 
 +
 
 +
So we need the new initscripts-8.45.44.tar.bz2 & 0001-Fix-comments-in-sysctl.conf.ppc-862597.patch, and the updated 0002-create-tmp-.X11-unix-in-rc.sysinit.patch
 +
 
 +
 
 +
=== Step 2: Integration : local mock build, can be done by a developer ===
 +
 
 +
* Remove the obsolete files
 +
 
 +
* Update the spec file.
 +
Version
 +
Release
 +
Patches
 +
Check the description, don't copy the 'Centos' wording to the SME package
 +
Copy the latest upstream Changelog
 +
Add/modify the changelog regarding SME patches
 +
 
 +
* Copy the new patch & source to the local directory
 +
 
 +
* Check that the patches apply cleanly (You don't want to see lines like Hunk #1 succeeded at 890 (offset 1 line).)
 +
make prep
 +
 
 +
* Try to build it
 +
make mockbuild
 +
 
 +
{{Note box|msg=Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance.
 +
* IF 32 BITS SYSTEM:
 +
BUILDARCH=i386 make mockbuild
 +
* IF 64 BITS SYSTEM:
 +
BUILDARCH=x86_64 make mockbuild}}
 +
* Once it builds cleanly on local mock, commit the changes to CVS. Note that you cannot submit the build yet as it will fail due to the new source.
 +
make commit
 +
 
 +
=== Step 3: Buildsys access and permissions needed ===
 +
 
 +
Then add new sources to buildsys ON BUILDSYS
 +
 
 +
If the above step has already been done then can update to latest in CVS, and retrieve the source tarball.
 +
 
 +
* get date for import
 +
$ date +%s
 +
1382918459
 +
 
 +
* fix name/version/release and date in the import.log (I don't have a good explanation for this)
 +
vi import.log*
 +
 
 +
* import the new source/tarball
 +
make new-sources FILES=*bz2
 +
 
 +
* Finally commit changes to CVS and build it.
 +
make commit tag build
 +
 
 +
=== Creating a new branch, eg SME 9 ===
 +
 
 +
== Package Build Steps ==
 +
check out package
 +
cp -a sme8 sme9
 +
rm -rf sme9/CVS
 +
cd sme9
 +
make prep
 +
 
 +
mv {sourcedir} {newsourcedir}
 +
* example e-smith-base-5.2.0 to e-smith-base-5.4.0
 +
rm -f *.patch
 +
* apply any fixes from spec to tree, only for SME Server packages
 +
* move lib/perl5/site_perl to share/perl5/vendor_perl
 +
tar Jcvf {newsource}.tar.xz {newsourcedir}
 +
 
 +
vi *.spec
 +
* remove patches and fixes applied directly to tree
 +
* bump version and reset release to 1
 +
* add changelog saying rolling for sme9
 +
make mockbuild
 +
* if this works then we can proceed
 +
rpm -qpl {package-version}_sme/{rpmname}.rpm
 +
* check that files look similar to the sme8 package
 +
 
 +
 
 +
=== These following steps need developer access ===
 +
They are here for reference, and transparency. They will need to be run on SME Server's build server.
 +
You do not need to do this part.  (If you have CVS access but not access to the build servers please don't run any of these commands)
 +
 
 +
cd ..
 +
cvs add sme9
 +
cd sme9
 +
 
 +
date +%s
 +
* get date for import
 +
vi import.log*
 +
* fix name/version/release and date
 +
cvs add .cvsignore *.spec Makefile import.log sources
 +
* be sure to add any other needed packages
 +
make new-sources FILES=*xz
 +
 
 +
make commit tag build
 +
 
 +
== Older notes ==
 +
* You can find more example  [[Rpm_build_example_using_cvs/mock|of how using Mock]]
 +
* To set up mock as used above see [http://wiki.contribs.org/Setting_up_RPM_Building_for_SME_Server Building using Mock]
 +
* To checkout packages from CVS see [[Simple_Package_Modification|Simplest way to develop patches to submit]]
 +
 
 +
* If you have problems on setting up the environment, ask on IRC or Devinfo.
  
 
Notes:
 
Notes:
  
 +
* Using the provided .cfg files I have verified smeserver-8-x86_64-base.cfg & smeserver-9-x86_64-base.cfg and managed to build e-smith-runit for SME 9
 
* the chroot environments are cached, but rebuilding them and checking for updates can be pretty network intensive, so you might want to consider setting up a local repository to pull from. mrepo (available from rpmforge) is pretty good for that.
 
* the chroot environments are cached, but rebuilding them and checking for updates can be pretty network intensive, so you might want to consider setting up a local repository to pull from. mrepo (available from rpmforge) is pretty good for that.
−
*there don't seem to be any hooks in mock to allow you to sign packages you've built, so if you do want signed packages you need to sign them afterwards via a rpm --resign $RPMS.
+
* there don't seem to be any hooks in mock to allow you to sign packages you've built, so if you do want signed packages you need to sign them afterwards via a rpm --resign $RPMS.
−
 
+
* If you get an error message 'create archive failed cpio: Bad magic' then try 'yum update' as this appears to be a known problem on COS6.
  
−
<noinclude>[[Category:Howto]]</noinclude>
+
[[Category:SME Server Development Framework]]
 +
[[Category:Development Tools]]
 +
[[Category:SME9-Development]]

Latest revision as of 09:57, 8 October 2021

PythonIcon.png Skill level: Developer
Risk of inconsistencies with Koozali SME Server methodology, upgrades & functionality is high. One must be knowledgeable about how changes impact their Koozali SME Server. Significant risk of irreversible harm.



Setting up RPM Building for SME Server

http://www.openfusion.net/linux/mocking_rpms

Mock is a Fedora project that allows you to build RPM packages within a chroot environment, allowing you to build packages for other systems than the one you're running on (e.g. building CentOS 4 32-bit RPMs on a CentOS 5 64-bit host), and ensuring that all the required build dependencies are specified correctly in the RPM spec file. You can find more example of how using Mock

Prerequisites

Warning.png Warning:
If you plan to build for sme10, you must use an el7 version of centos minimal. It is also workable on a Fedora25


Installation CentOS6 / CentOS7 minimal

Just install a minimal el6 installation. You can use either the DVD, the minimal CD install, a net install with PXE, it's up to you. Of course if you prefer you can use a system with a Xorg-server installed (eg: gnome, kde, xfce...)

  • Setup network:

To start the networking 

[root@buildcos9 ~]# /etc/sysconfig/network-scripts/ifup-eth eth0
  • Set DHCP to install basic utilities
[root@buildcos9 ~]#  dhclient eth0
  • Install utilities
[root@buildcos9 ~]# yum install nano system-config-network-tui mc man htop top iftop iptraf screen openssh-clients mlocate
  • Finalise network setup
[root@buildcos9 ~]# nano /etc/sysconfig/network-scripts/ifcfg-eth0

and set manually: 

DEVICE="eth0"
BOOTPROTO="none"
HWADDR="00:E0:4C:68:21:6D"
IPADDR=192.168.xxx.12
NETMASK=255.255.255.0
GATEWAY=192.168.xxx.3
DNS1=192.168.xxx.3 
IPV6INIT=no
NM_CONTROLLED="yes"
ONBOOT="YES"
TYPE="Ethernet"
UUID="fe6ebc0d-dbee-4bf6-a3cf-5e84c5453549"

Check and restart: 

[root@buildcos9 ~]# chkconfig network on
[root@buildcos9 ~]# service network restart
  • Enable SSH

SSH is present but is disabled at this stage. To enable ssh issue: 

service sshd start

You may have a slow login to ssh, if needed : 

nano /etc/ssh/sshd_config

And change the “UseDNS yes” field to “UseDNS no”. Now change the “GSSAPIAuthentication yes” line to “GSSAPIAuthentication no”.

You have to restart the ssh service or the server

  • Reboot system to ensure all new config are permanent
 [root@buildcos9 ~]# reboot
  • Disable SELinux SME9

It will be easier to see what's going on (and turn off selinux at kernel level, just to be sure) 

[root@buildcos9 ~]# sed -i -e 's/rhgb quiet/selinux=0/g' /boot/grub/grub.conf
[root@buildcos9 ~]# sed -i -e 's/SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config
  • SME10

in /etc/selinux/config set */SELINUX=disabled

  • Remove selinux-policy-targeted and authconfig -they conflict with some e-smith/smeserver packages
[root@buildcos9 ~]# yum remove selinux-policy-targeted authconfig
  • Apply all available updates.
yum update
  • You need to have EPEL repository configured as EPELs version of mock will be used
# Enable EPEL for Centos 6
rpm -Uvh http://fr2.rpmfind.net/linux/epel/6/i386/epel-release-6-8.noarch.rpm
  1. Enable EPEL and sclo repo for Centos 7

rpm -ivh https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-12.noarch.rpm or (works 29Dec19 - bjr) 

yum install -y epel-release

rpm -ivh http://mirror.centos.org/altarch/7/extras/aarch64/Packages/centos-release-scl-rh-2-3.el7.centos.noarch.rpm 

yum install centos-release-scl-rh
  • You need to create a new user, for mock will not accept user root for building packages
useradd <username>
passwd <username>

The new user needs to be granted sudo rights 

yum install sudo
/usr/sbin/usermod -aG wheel <username>

Edit the file /etc/sudoers: 

nano /etc/sudoers

go down and find the following lines: 

 ## Allows people in group wheel to run all commands
 # %wheel	ALL=(ALL)	ALL

and remove the hash in front of %wheel so it looks like... 

## Allows people in group wheel to run all commands
%wheel	ALL=(ALL)	ALL

Log out as root and login as <username>


So then the installation of mock is just:

Installing Mock

Important.png Note:
Mock creates chroots and builds packages in them. Its only task is to reliably populate a chroot and attempt to build a package in that chroot. Mock is currently being used for all Fedora builds. Mock takes a srpm and builds it in a chroot. This ensures that your BuildRequires lines are correct, that there are no missing dependencies, and that it builds cleanly. The Mock project page has more information on Mock.


Install mock, make and rpm-build and python-ctypes packages (the latter for better setarch support) 

sudo yum --enablerepo=epel install mock python-ctypes glances
sudo yum install make rpm-build rpm-sign wget

Add yourself to the 'mock' group that will have now been created 

sudo /usr/sbin/usermod -G mock <username>

The mock package creates an /etc/mock directory with configs for various OS versions (mostly Fedoras). dani has provided working configurations for SME Server here http://mirror.contribs.org/contribs/dani/mock/

Copy all the .cfg files into /etc/mock including overwriting the site-defaults.cfg

  • Remove dup file as root
[root@buildcos9 mock]# cd /etc/mock/

[root@buildcos9 mock]# rm site-defaults.cfg

wget http://mirror.contribs.org/contribs/dani/mock/site-defaults.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-i386-base.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-i386-iso.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-x86_64-base.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-8-x86_64-iso.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-i386-base.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-i386-iso.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-x86_64-base.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-9-x86_64-iso.cfg
wget http://mirror.contribs.org/contribs/dani/mock/smeserver-10-x86_64-base.cfg
  • Change group ownership to mock:
[root@buildcos9 mock]# chown root:mock /etc/mock/*
  • workaround (as of 11Oct2016 - this does not seem necessary - the required line is already in the cfg files - bjr)

if the tag 'sme' is not added to the built package see bugzilla:8636 and add

to smeserver-8-i386-base.cfg & smeserver-8-x86_64-base.cfg 

config_opts['macros']['%dist'] = ".el5.sme"

to smeserver-9-i386-base.cfg & smeserver-9-x86_64-base.cfg 

config_opts['macros']['%dist'] = ".el6.sme"
  • Script to create build directories
#!/bin/sh
mkdir ~/rpmbuild
for i in BUILD BUILDROOT RPMS SOURCES SPECS SRPMS
do
 mkdir ~/rpmbuild/$i
done

Testing building environment

You can then run a test by doing: 

# e.g. initialise a smeserver-9-x86_64-base chroot environment
[chris@build-system SRPMS] $ CONFIG=smeserver-9-x86_64-base
[chris@build-system SRPMS] $ mock -r $CONFIG --init

It might take a while to configure at the "Start: yum update"

which will setup an initial chroot environment using the given config. If that seemed to work (you weren't inundated with error messages), you can try a build

Perform a Build for sme9 64 bits from an SRPM

You will need an SRPM, for instance from http://mirror.contribs.org/releases/testing/9/smeupdates-testing/SRPMS/ you can use wget to download it in your terminal 

# Rebuild the given source RPM within the chroot environment
# usage: mock -r <mock_config> --rebuild /path/to/SRPM e.g.
 [chris@build-system SRPMS]$ CONFIG=smeserver-9-x86_64-base
 [chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/e-smith-runit-2.4.0-1.el6.sme.src.rpm

If the build succeeds, it drops your packages into the /var/lib/mock/$CONFIG/result directory. If it fails, you can check mock output, the *.log files above for more info, and/or rerun mock with the -v flag for more verbose messaging. If you get "warning: Could not canonicalize hostname: " it can be ignored. This is a DNS resolution error and the easy solution is to add your hostname to /etc/hosts 

ls -1 /var/lib/mock/smeserver-9-x86_64/result
build.log
e-smith-runit-2.4.0-1.el6.sme.noarch.rpm
e-smith-runit-2.4.0-1.el6.sme.src.rpm
root.log
state.log
Perform a Build for sme9 32 bits from an SRPM
[chris@build-system SRPMS]$ CONFIG=smeserver-9-i386-base
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/rpmbuild/SRPMS/e-smith-runit-2.4.0-1.el6.sme.src.rpm
Perform a Build for sme8 64 bits from an SRPM
[chris@build-system SRPMS]$ wget http://mirror.contribs.org/releases/8.0/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm 
[chris@build-system SRPMS]$ CONFIG=smeserver-8-x86_64-base
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm

see the built rpm in the relevant folder 

ls -1 /var/lib/mock/smeserver-8-x86_64/result
Perform a Build for sme8 32 bits from an SRPM
[chris@build-system SRPMS]$ CONFIG=smeserver-8-i386-base
[chris@build-system SRPMS] $ mock -r $CONFIG --rebuild ~/smeupdates-testing/SRPMS/perl-CGI-FormMagick-0.92-23.el5.sme.src.rpm

Install and config CVS

Important.png Note:
SME Server uses CVS for maintaining code and packages. SME Server core and contribs code is stored in CVS, this is a system to maintain version history and change management for program code.

There are two repositories hosting code for SME Server: 

   SME Server - holding the source code for the SME Server core
   SME Contribs - holding the source code for contribs (add-ons) for SME Server

Both repositories are organized by the package name under which they appear in the software installer in the server-manager (or yum). please see CVS page for more specific information


If this works, it is worth grabbing either CVS for one module, or the whole CVS tree using this : http://wiki.contribs.org/Simple_Package_Modification

  • Setup CVS to use ssh by creating /etc/profile.d/smebuild.sh with the following content
[chris@buildcos9 profile.d]$ sudo nano smebuild.sh

Paste this: 

# Developer environment
# This gets symlinked into /etc/profile.d

export CVS_RSH=ssh # tell CVS to use ssh

# DO NOT set CVSROOT

alias rm='rm -i'
alias cp='cp -i --preserve=timestamps'
alias mv='mv -i'

You have to logout and login again to the console for changes to take effects.

  • Check CVS account

You need top create an account on shell.koozali.org, then get an admin to include you in sme group + give access to Buildsys. Can be done later...Of course it is not an issue if you have not a cvs access like we will see below, you can retrieve package in mode anonymous.

KOOZALI CVS access

This chapter is used only by developers who aim to be granted in write access to the source code on shell.koozali.org, you don't need it if you want to only make patch or modify code in your build environment. You can see Package_Modification for more information.

  • SME Server code is stored in the CVS on shell.koozali.org. To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org. With this account the development team can give you access to the CVS repository.
  • After you have had your account created you can ask the development team to give you developer access to smecontribs. Create a bug in the Bug Tracker as usual.
  • If local username is different to koozali.org username edit ~/.ssh/config:
Host  shell.koozali.org
User  koozaliusername (without @shell.koozali.org)
Host  shell.koozali.org
User  koozaliusername (without @shell.koozali.org)
  • Don't forget to set the proper privileges on the file
chmod 600 ~/.ssh/config

the port 222 will need to be defined in ~/.ssh/config host shell.koozali.org port 222

you can also define there the rsa key you want to use...

If you have a number of different keys ssh/cvs may try and use the wrong one.

You can add the following to your ~/.ssh/config file to specify the correct key: 

Host  shell.koozali.org
User  koozaliusername
IdentityFile ~/.ssh/my.private.key

Using CVS - Getting source Code

We need to create directory in /home/chris for example this how you can do 

exchange  - used to move files around with wget
work - used for CVS jobs
rpmbuild - used to build with Mock
rpms - used for CVS tree download

[chris@buildcos9 work]$ cd /home/chris
[chris@buildcos9 work]$  mkdir {exchange,work,rpmbuild,rpms}

CVS is used for maintaining code and packages.

We first need to download (with CVS this is called retrieving) the code for the package to be modified from the SME Server CVS repository at Koozali.org (we can also retrieve the CVS for the smecontribs tree). Two cases with CVS:

To be able to work on your code in the SME Server CVS repository you need an account on Koozali.org. If you do not have an account at CVS, retrieving is performed as an anonymous user and changes cannot be saved.

Case 1 - Registered Users

- To retrieve a package or a full tree as a registered user, cd into a work directory and issue following commands:

  • for smeserver
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P packagename [individual package]
cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P rpms [full tree]
env CVS_RSH=ssh cvs -d:ext:YOURUSERNAME@shell.koozali.org:/cvs/smeserver checkout CVSROOT rpms common
  • for smecontribs
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P packagename [individual package]
cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P rpms [full tree]
env CVS_RSH=ssh cvs -d:ext:YOURUSERNAME@shell.koozali.org:/cvs/smecontribs checkout CVSROOT rpms common
Case 2 - Anonymous users

- To retrieve a package or a full tree as an anonymous user, cd into a work directory and issue following commands:

  • for smeserver
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smeserver checkout -P packagename [individual package]
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smeserver co -P rpms [full tree]
  • for smecontribs
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smecontribs checkout -P packagename [individual package]
cvs -z3 -d:pserver:anonymous@shell.koozali.org:/cvs/smecontribs co -P rpms [full tree]

Note: If at a later stage you obtain a valid account with CVS, you will need to edit ALL packages /CVS/Root in order to save your work in CVS. As an example, for package "packagename", CVS/Root will need to be edited to be exactly as shown below: 

[chris@buildcos9 work]$ cat packagename/CVS/Root
:ext:shell.koozali.org:/cvs/smeserver

ok - CVS/Root uses 'ext' which should be SSH access the port 222 will need to be defined in ~/.ssh/config host shell.koozali.org port 222 you can also define there the rsa key you want to use...

Package modification

  • Prepare work areas, make some directories:
[chris@buildcos9 ~]$ pwd
/home/chris
  • [chris@buildcos9 ~]$ ls
exchange  rpmbuild  rpms  test  work
  • Import cvs in your workspace
[chris@buildcos9 ~]$ cd work/
[chris@buildcos9 ~]$ mkdir smeserver
[chris@buildcos9 ~]$ cd smeserver
[chris@buildcos9 work]$ cvs -z3 -d:ext:shell.koozali.org:/cvs/smeserver co -P packagename

or for smecontribs 

[chris@buildcos9 ~]$ cd work/
[chris@buildcos9 ~]$ mkdir smecontribs
[chris@buildcos9 ~]$ cd smecontribs
[chris@buildcos9 work]$ cvs -z3 -d:ext:shell.koozali.org:/cvs/smecontribs co -P packagename
  • To refresh run the following from the rpms directory, or any lower directory with a CVS dir.
[chris@buildcos9 ~] cd packagename/
[chris@buildcos9 packagename]$ cvs update -dPA
  • Change to work directory, in this example, we are working on sme9:
[chris@buildcos9 packagename] cd sme9
  • To prepare a tree
[chris@buildcos9 sme9]$  cvs update -dPA [if not done previously]
[chris@buildcos9 sme9]$  make clean  [ignore errors about tar.xz:file missing]     
[chris@buildcos9 sme9]$  make prep
  • Make a patch

Our patch will be a done with the 'diff' tool which compares the edited files against the original files.

  • We need to to save a copy of the original files:
cp -R packagename-1.2.3 packagename-1.2.3.old
  • Switch to the tree and make required changes/modifications.
  • Create a patch using a sensible name, eg "packagename-1.2.3-migrate_to_UNC_syntax.patch":
[chris@buildcos9 sme9]$ diff -urN packagename-1.2.3.old packagename-1.2.3 > packagename-1.2.3-your_changes-description.patch
  • Update the spec file
[chris@buildcos9 sme9]$ nano packagename.spec
  • Increase the release version at the beginning of the spec file by 1:
%define release 4
  • Add the patch to the existing patchlist section not forgetting to increase the patch number by 1:
Patch3: packagename-1.2.3-my_changes_description
  • Add patch version to the %setup section not forgetting to increase number by 1:
%patch3 -p1
  • Update the changelog section, including the release version and bug number:
%changelog
* Mon Apr 01 2013 name <email> 1.2.3-4.sme
- Summary of changes [SME: xxxx]

NOTE: The formatting of date information is critical, any errors will prevent successful commit to CVS. In doubts, check previous entries in the changelog.

NOTE: There must be a bug reference for every change, it must be in the format [SME: xxxx]

  • Check changes against CVS
[chris@buildcos9 sme9]$ cvs diff -u packagename.spec
  • Build the test RPM
[chris@buildcos9 sme9]$ pwd
/home/chris/work/packagename/sme9

[chris@buildcos9 sme9]$ make mockbuild


Important.png Note:
Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance.
  • Default
make mockbuild
  • IF 32 BITS SYSTEM:
BUILDARCH=i386 make mockbuild
  • IF 64 BITS SYSTEM:
BUILDARCH=x86_64 make mockbuild


Basic Testing of Package:

Check for built packages: 

[chris@buildcos9 sme9]$ ls -la packagename-1_2_3-4_el6_sme

- Send new package (and patch if required) to workstation for testing/processing, as an example: 

[chris@buildcos9 packagename-1_2_3-4_el6_sme]$ scp -P 22 packagename-1_2_3-4_el6_sme.noarch.rpm chris\@192.168.0.1:/home/chris/WIP

- Move package to an ibay on test server

- Install package (yum localinstall or rpm -Uvh, for SME9 you can also use yum update packagename-1_2_3-4_el6_sme.noarch.rpm)

- Ensure nothing is broken.


Notes on mock/cvs

Normally you make builds from the working directory of the module you are working on, eg ~/rpms/e-smith-runit/sme9: 

# Build the given module within the chroot environment
$ cd ~/rpms/e-smith-runit/sme9
$ make mockbuild

# If there is no sme9 directory do the following (to test ability to build only)
$ cd ~rpms/module_name
$ ln -s sme8 devel
$ cd devel
$ make mockbuild

It will determine if this is for SME Server 9 or 8, and the package to build, based on which directory that you are in. The results are put under that directory. 

ls -1 ~/rpms/e-smith-runit/sme9/e-smith-runit-2_4_0-2_el6_sme/
build.log
e-smith-runit-2.4.0-2.el6.sme.noarch.rpm
e-smith-runit-2.4.0-2.el6.sme.src.rpm
root.log
state.log

Note if there are any perl directory requirments : 

$ rpm -qpl package_name.noarch.rpm |grep perl
Finalising in CVS and buildsys
  • Add the patch file to cvs:
[chris@buildcos9 sme9]$ cvs add packagename-1.2.3-your_changes-description.patch
cvs add: scheduling file `packagename-1.2.3-your_changes-description' for addition
cvs add: use 'cvs commit' to add this file permanently

NOTE: ignore this suggestion! Just proceed to next stage or you will end up in all kinds of problems....

  • Build package on Shad's system:
[chris@buildcos9 sme9]$ make commit tag build

Note: this can also be done in stages: 

make commit
make tag
make build
  • Check for Build:
http://buildsys.contribs.org/plague/
  • Check for emails received from buildsys@contribs.org, you should get one from CVS and two from the Buildsystem.
  • If (and only if) build has been successful, open relevant ticket in Bugzilla and notify of build:

Fixed in packagename: 

%changelog
* Mon Apr 01 2013 name <email> 1.2.3-4.sme
- Summary of changes [SME: xxxx]

You may also wish to provide some tips about verification.

Patching an upstream package

initscripts is an example of a package where we have to patch the upstream package. Note that this is not the best approach, new method to be documented by Ian soon.

Original author Ian Wells

Here it is a 3-step process, Analysis, Integration, buildsys update. The majority of the work is in steps 1 & 2 where there needs to be an analysis of what has changed, and then an integration stage where you need to check that the patches still work, update CVS with new and modified files etc. Finally the new source needs updating into the buildsys before the official build will work. 

initscripts-8.45.44-3.el5.centos.src.rpm

Steps to update initscripts

Step 1: Analysis, by anyone

Check the contents of the SRPMs 

$ rpm -qlp initscripts-8.45.44-3.el5.centos.src.rpm
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
initscripts-8.45.44.tar.bz2
initscripts-centos-branding.patch
initscripts.spec

$ rpm -qlp initscripts-8.45.42-2.1.el5.sme.src.rpm
0002-create-tmp-.X11-unix-in-rc.sysinit.patch
initscripts-8.45.38-smepatches.patch
initscripts-8.45.42-kpartx.patch
initscripts-8.45.42.tar.bz2
initscripts-centos-branding.patch
initscripts.spec


Check what is different (do a directory compare)


  • Identical
initscripts-centos-branding.patch (content is identical, although files are not)
  • Modified
initscripts.spec
0002-create-tmp-.X11-unix-in-rc.sysinit.patch (patch is identical but line numbers changed)
  • New
initscripts-8.45.44.tar.bz2 (New source will need to be added to buildsys)
0001-Fix-comments-in-sysctl.conf.ppc-862597.patch (will be added to CVS)
  • Obsolete
initscripts-8.45.42-kpartx.patch (can be removed from CVS)
initscripts-8.45.42.tar.bz2
  • Not in upstream SRPM as this is an SME Server patch to be applied each time
initscripts-8.45.38-smepatches.patch


So we need the new initscripts-8.45.44.tar.bz2 & 0001-Fix-comments-in-sysctl.conf.ppc-862597.patch, and the updated 0002-create-tmp-.X11-unix-in-rc.sysinit.patch


Step 2: Integration : local mock build, can be done by a developer

  • Remove the obsolete files
  • Update the spec file.
Version
Release
Patches
Check the description, don't copy the 'Centos' wording to the SME package
Copy the latest upstream Changelog
Add/modify the changelog regarding SME patches
  • Copy the new patch & source to the local directory
  • Check that the patches apply cleanly (You don't want to see lines like Hunk #1 succeeded at 890 (offset 1 line).)

make prep

  • Try to build it
make mockbuild


Important.png Note:
Mock will determine which configuration to use, but you can specify it if wanting to build for a different architecture for instance.
  • IF 32 BITS SYSTEM:
BUILDARCH=i386 make mockbuild
  • IF 64 BITS SYSTEM:
BUILDARCH=x86_64 make mockbuild


  • Once it builds cleanly on local mock, commit the changes to CVS. Note that you cannot submit the build yet as it will fail due to the new source.

make commit

Step 3: Buildsys access and permissions needed

Then add new sources to buildsys ON BUILDSYS

If the above step has already been done then can update to latest in CVS, and retrieve the source tarball.

  • get date for import
$ date +%s
1382918459
  • fix name/version/release and date in the import.log (I don't have a good explanation for this)
vi import.log*
  • import the new source/tarball
make new-sources FILES=*bz2
  • Finally commit changes to CVS and build it.
make commit tag build

Creating a new branch, eg SME 9

Package Build Steps

check out package 

cp -a sme8 sme9
rm -rf sme9/CVS
cd sme9
make prep

mv {sourcedir} {newsourcedir}
  • example e-smith-base-5.2.0 to e-smith-base-5.4.0
rm -f *.patch
  • apply any fixes from spec to tree, only for SME Server packages
  • move lib/perl5/site_perl to share/perl5/vendor_perl
tar Jcvf {newsource}.tar.xz {newsourcedir}

vi *.spec
  • remove patches and fixes applied directly to tree
  • bump version and reset release to 1
  • add changelog saying rolling for sme9
make mockbuild
  • if this works then we can proceed
rpm -qpl {package-version}_sme/{rpmname}.rpm
  • check that files look similar to the sme8 package


These following steps need developer access

They are here for reference, and transparency. They will need to be run on SME Server's build server. You do not need to do this part. (If you have CVS access but not access to the build servers please don't run any of these commands) 

cd ..
cvs add sme9
cd sme9

date +%s
  • get date for import
vi import.log*
  • fix name/version/release and date
cvs add .cvsignore *.spec Makefile import.log sources
  • be sure to add any other needed packages
make new-sources FILES=*xz

make commit tag build

Older notes

  • If you have problems on setting up the environment, ask on IRC or Devinfo.

Notes:

  • Using the provided .cfg files I have verified smeserver-8-x86_64-base.cfg & smeserver-9-x86_64-base.cfg and managed to build e-smith-runit for SME 9
  • the chroot environments are cached, but rebuilding them and checking for updates can be pretty network intensive, so you might want to consider setting up a local repository to pull from. mrepo (available from rpmforge) is pretty good for that.
  • there don't seem to be any hooks in mock to allow you to sign packages you've built, so if you do want signed packages you need to sign them afterwards via a rpm --resign $RPMS.
  • If you get an error message 'create archive failed cpio: Bad magic' then try 'yum update' as this appears to be a known problem on COS6.
Retrieved from "https://wiki.koozali.org/index.php?title=Setting_up_RPM_Building_for_SME_Server&oldid=40493"