Difference between revisions of "Build an Official ISO"

	Skill level: Developer
	Risk of inconsistencies with Koozali SME Server methodology, upgrades & functionality is high. One must be knowledgeable about how changes impact their Koozali SME Server. Significant risk of irreversible harm.

	Work in Progress:
	This page is a Work in Progress. The contents off this page may be in flux, please have a look at this page history the to see list of changes.

Introduction

These Notes have been kindly shared by Ian Wells, thanks to him. This howto is there to document the way on the build of an official Iso. For an introduction you must follow first this howto and get a full workable buildserver

~~bugzilla:7675~~ build_installer and build_ISO are meant to be run in a mock chroot with the stage tree nfs mounted into it. The build box can be the same, eg COS6 x64 for all ISOs, it is only the mock image that needs to match the ISO, and this is magically handled by the mock configs. You also need to have access to the extras repo as many of the packages needed to build the installer and ISO are contained in there. This repo is to house the needed packages that aren't going to be included in the ISO. For clarification read http://bugs.contribs.org/show_bug.cgi?id=7675#c37

~~bugzilla:7931~~ Please read Bug #7931 regarding repodata and symlinks.

	Warning:
	due to the recent change of some plugins, this howto is only workable now, with the Mock Version: 1.1.41 --Stephdl (talk) 21:40, 16 March 2015 (CET) du to the work of dani, we should use theversion of mock > 1.2 --Stephdl (talk) 11:06, 29 June 2015 (CEST)

	Warning:
	build of iso on cluster should be done on builder7 with releases user

Folder structure, both local and buildsys

This document presumes that

~/builds_bin is builds_bin from CVS
~/cdrom.image is cdrom.image from CVS
~/anaconda-po contains the anaconda translations
~/smeserver contains a local smeserver mirror 
~/smeserver/rpms is rpms folder from CVS

if absent then :

cd ~ 
mkdir -p smeserver/builds_bin smeserver/cdrom.image smeserver/cdrom.image
ln -s smeserver/builds_bin builds_bin 
ln -s smeserver/cdrom.image cdrom.image
mkdir anaconda-po

then update content or populate

cd smeserver
cvs -z3 -d:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver co -P builds_bin
cvs -z3 -d:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver co -P cdrom.image
cvs -z3 -d:pserver:anonymous@smeserver.cvs.sourceforge.net:/cvsroot/smeserver co -P rpms
cd ~

Local Only: mock requirement

on centos 7

yum install python-decoratortools

Local Only: mock plugin : iso_prepare

You have to install a new plugin for mock : Iso_prepare

copy the content of http://bugs.contribs.org/attachment.cgi?id=5145 and paste it in

sudo vim /usr/lib/python2.6/site-packages/mockbuild/plugins/iso_prepare.py

on centos 7

sudo vim /usr/lib/python2.7/site-packages/mockbuild/plugins/iso_prepare.py

then you will need to comment the two follwoing lines (32 and 33) if you build on a centos 7 box:

#        for l in glob.glob("/dev/loop*"):
#            buildroot.mounts.add(BindMountPoint(srcpath=l, bindpath=buildroot.make_chroot_path(l)))

Local Only: update_repos

The script update_repos relies on RPM2 which can be found as a RPM.

yum install perl-RPM2 repoview --enablerepo=epel

update_repos (/build/smeserver/stage/bin/update_repos) is used to keep the repos in sync and to create the ‘stage’ from which the ISO is built. On the local build system update_repos can be run manually before building an ISO, whereas it runs every hours on the buildsys.

	Note:
	Note that update_repos copies packages from smeupdates-testing to stage but not from smetest, so packages freshly built won't be included into the new ISO until they are manually moved from smetest to smeupdates-testing

The buildsys has two main folders that are relevant for building an ISO.

repo      => '/build/smeserver/repo/ ', eg '/build/smeserver/repo/testing/9/'
stage     => '/build/smeserver/stage/', eg  '/build/smeserver/stage/9/'

repo is the folder that replicates to the mirrors. Be careful of all changes here.
stage is the working directory. Note that update_repos expects the folder structure to be in place.

It also needs all GPG keys for every package that it sees, it can take a while to find them all.

Local Only: Create the repo folder and the binary program folder

Local Only

mkdir -p /build/smeserver/repo
mkdir -p /build/smeserver/stage/bin

The content of builds_bin from CVS needs to be copied to /build/smeserver/stage/bin

/bin/cd ~/builds_bin
/usr/bin/cvs update -dPA
/bin/cp ~/builds_bin/* /build/smeserver/stage/bin

Local Only: Mock Special Settings

hard-coded UID of release account

verify that in all /etc/mock/smeserver-{i386,x86_64}-iso.cfg you have config_opts['chrootuid'] = os.getuid()

- config_opts['chrootuid'] = 10020
+ config_opts['chrootuid'] = os.getuid()

	Note:
	Due to local settings you have to edit each configuration of mock and edit smeserver-X-{i386,x86_64}-iso.cfg files to include /build in the chroot

These are the changes that you need to made if you do not have the nfs mounts on your local machine

config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/export/build', '/build', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/mirrors', '/mirrors', 'nfs', 'defaults,noatime,nodiratime,nosuid'))

with

#config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/export/build', '/build', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
#config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/mirrors', '/mirrors', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/build', '/build' ))

Eg:

sudo vim /etc/mock/smeserver-9-x86_64-iso.cfg

Local Only: Patch a Mock Plugin

There is also an update to /usr/lib/python2.6/site-packages/mockbuild/mounts.py that needs to happen so loop devices get mounted correctly. Check /var/lib/mock/smeserver-VERSION-ARCH/result/root.log after mock -r smeserver-VERSION-ARCH-iso --init (eg mock -r smeserver-8-i386-iso --init). The mounting of the /dev/loopX should pass (return code 0). If it fails (return code 32) then the patch below is needed on your build host. see http://bugs.contribs.org/show_bug.cgi?id=7675#c61

save as mounts.py.20150626.daniel.berteaud.patch see http://bugs.contribs.org/attachment.cgi?id=5146

--- /usr/lib/python2.6/site-packages/mockbuild/mounts.py	2015-06-04 13:34:04.000000000 +0200
+++ mounts.py	2015-06-26 18:05:53.924416540 +0200
@@ -76,6 +76,9 @@
 
    @traceLog()
    def mount(self):
+        if not os.path.isdir(self.srcpath) and not os.path.isfile(self.srcpath):
+            util.touch(self.bindpath)
+
        if not self.mounted:
            cmd = ['/bin/mount', '-n',
                   '--bind', self.srcpath, self.bindpath ]
@@ -126,5 +129,9 @@
            m.umount()

    @traceLog()
+    def get_mounted(self):
+        return [ m.mountpath for m in self.mounts if m.ismounted() ]
+
+    @traceLog()
    def get_mountpoints(self):
        return [ m.mountpath for m in self.mounts ]

then do

sudo patch -N  /usr/lib/python2.6/site-packages/mockbuild/mounts.py < mounts.py.20150626.daniel.berteaud.patch

save as mount.py.20150626.daniel.berteaud.patch see http://bugs.contribs.org/attachment.cgi?id=5147

--- /usr/lib/python2.6/site-packages/mockbuild/plugins/mount.py	2015-06-04 13:34:04.000000000 +0200
+++ mount.py	2015-06-26 18:05:53.921418037 +0200
@@ -41,11 +41,11 @@
        self.opts = conf
        plugins.add_hook("preinit", self._mountPreInitHook)
        for device, dest_dir, vfstype, mount_opts in self.opts['dirs']:
-            builroot.mounts.add(FileSystemMountPoint(buildroot.make_chroot_path(dest_dir),
+            buildroot.mounts.add(FileSystemMountPoint(buildroot.make_chroot_path(dest_dir),
                                                    filetype=vfstype,
                                                    device=device,
                                                    options=mount_opts))
    @traceLog()
    def _mountPreInitHook(self):
        for device, dest_dir, vfstype, mount_opts in self.opts['dirs']:
-            mockbuild.util.mkdirIfAbsent(self.builroot.make_chroot_path(dest_dir))
+            mockbuild.util.mkdirIfAbsent(self.buildroot.make_chroot_path(dest_dir))

then do

sudo patch -N  /usr/lib/python2.6/site-packages/mockbuild/plugins/mount.py < mount.py.20150626.daniel.berteaud.patch

Preparation for a new ISO release

Generate a new Changelogs

When generating an ISO the Release Notes for the ISO should contain the full delta from the last ISO. Ian gets a script for doing that

The Smeserver version name

The Version number comes from a rpm, smeserver-release, you need to bump the version in the spec file and build the rpm, think to push it to smeupdates-testing.

RPM dependencies

The smeos, smeupdates, and smeupdates-testing repos are included so we can stage the packages we are going to release without actually having to release them. Normally the smeupdates-testing repo is used for verification (testing is for developer testing). During the ISO release schedule (usually very short) we hold moving packages to updates-testing until the ISO is built.

Examples of dependencies

cp /mirrors/rpmforge/redhat/el6/en/i386/dag/RPMS/perl-Taint-Util-0.08-1.el6.rf.i686.rpm /build/smeserver/repo/testing/9/smetest/i386/RPMS/

cp /mirrors/centos/6/os/i386/Packages/libtevent-0* /build/smeserver/repo/testing/9/smeupdates-testing/i386/RPMS/

This shows that on the buildsys the /mirrors has the main upstream repos and they will normally hold the RPM needed to resolve the dependency. You need to get it into smeupdates-testing, but it is safer to first copy the RPM to smetest and check that all is well.

Rpm update cycle

If an RPM is released during the update cycle:

smetest->smeupdates-testing->smeupdates

If an RPM is only needed for a new ISO and never needed as an update (before the ISO is released) :

/mirrors->smeupdates-testing->smeos

Note that you don't put it into smeos, that is taken care of by the build scripts.

What are smeos/smeextras

Every package that needs to be in the ISO will go into smeos, by the buildsys.
Packages that are needed for building the ISO, but not actually needed in the ISO, will be in smeextras

How get a Fully updated ISO

If your Iso is not fully updated and a 'yum update' fetch updates after the first boot, in generally centos updates. You will need to push missed updates from smetest to smeupdates-testing.

RPM QA verification

	Warning:
	During the ISO release schedule, ALL RPMS MUST BE VERIFIED IN SMEUPDATES-TESTING.

Server-Manager translations

Check translations from https://translate.koozali.org/patches/ if the filesize is >0 then need to patch smeserver-locale (to sme8 & sme9) and produce new rpms that will must be pushed to smeupdate-testing

Anaconda translations

Check installer translations from https://translate.koozali.org/anaconda/

get the latest files (-w 1 means wait one second between two requests, to prevent your IP behing blacklisted):

wget --mirror --no-parent -w 1 -A.mo -P ~/anaconda-po https://translate.koozali.org/anaconda/

Check if installer needs re-building

	Warning:
	If either anaconda has changed OR there is new content in cdrom.image (apart from the release note) then the installer needs to be rebuilt (by the script build_installer). Otherwise just the ISO needs to be built (by the script Build_ISO).

Update cdrom.image content

Update cdrom.image to the latest in CVS

create a new tree if this is a new SME version

cd smeserver
cvs -z3 -d:ext:smeserver.cvs.sourceforge.net:/cvsroot/smeserver co -P cdrom.image
cd cdrom.image 
cp -r sme9 sme10
rm -rf sme10/CVS
rm -rf sme10/isolinux/CVS
rm -rf sme10/Packages/base/CVS  
rm -rf sme10/Packages/CVS
rm -rf sme10/product/CVS
rm -rf sme10/product/installclasses/CVS
rm -rf sme10/product/pixmaps/CV
rm -rf sme10/updates/CVS
cvs add sme10
cd sme10

add new key

wget http://mirror.canada.pialasse.com/releases/testing/10/smeos/x86_64/RPM-GPG-KEY-koozali

do a little tidy up, then

find ./ -name CVS -prune -o -print | xargs cvs add
cvs commit -m 'new cdrom.image version for sme10'

alternatively to only update content

cd ~/cdrom.image 
cvs update -dPA

Check Anaconda version

Check which anaconda was used upstream, (eg http://vault.centos.org/5.11/os/SRPMS/ & http://vault.centos.org/6.6/os/Source/SPackages/) and is it newer than the one in CVS. If it is then anaconda needs to be updated and patched. | The latest Anaconda also needs to be in smeextras.

Bump smeserver-release if needed

Most of the time, we'll create a new ISO when we release a new SME Server version. So, we have to bump the version of the smeserver-release RPM.

	Note:
	Be careful to bump the version, the release and the displayversion macro, eg %define version 8.2 %define release 0.rc2 %define displayversion %{version}rc2

Bump version in the splash screen

This is only needed on SME9 and SME10: we need to update the title of the splsh screen. For this, we have to update the title line in cdrom.images/sme9/isolinux/isolinux.cfg, eg

menu title Welcome to SME Server 9.1 beta3

Remove the 'Beta warning' in Anaconda

For SME10

edit the file .buildstamp

cd /build/smeserver/stage/extra/images/10/product
vim .buildstamp

[Main]
Product=Koozali SME Server
Version=10alpha2
BugURL=https://bugs.koozali.org
IsFinal=False
UUID=alpha2.x86_64
[Compose]
Lorax=19.6.66-1

change isFinal to True

Also bump the Version !

For SME9

The Iso gets a 'beta' tag with warnings during the install process. How remove the beta tag, add --final in the /build/smeserver/stage/bin/build_installer

# add --final when we have final product
  PYTHONPATH=/usr/lib/anaconda/ sudo /usr/lib/anaconda-runtime/buildinstall \
      --product "$distname" \
      --version "${distvers/[^0-9.]*/}" \
      --release "$distname" \
      --brand "centos" \
      --final \
      --bugurl http://bugs.contribs.org/ \
      --output $(pwd)/$distarch/ \
      /build/smeserver/repo/$distrel/smeextras/$distarch/ \
      $(pwd)/$distarch/
  /bin/sed -i -e "s|packagedir =.*|packagedir = Packages|" $(pwd)/$distarch/.treeinfo

For SME8

Edit cdrom.image/sme8/updates/installclasses/smeinstallclass.py and remove the "betanag" entry from the dispatch.setStepList list

   def setSteps(self, dispatch):
       dispatch.setStepList(
                "betanag",  <- remove this line
                "language",
                "keyboard",
                "findrootparts",

Prepare SME Server 10

Local Only: Update local mirror as seen by the chroot

Local Only

rsync -av --delete --progress ~/smeserver/releases/10/ /build/smeserver/repo/10

Create release specific folder and link (or rename the folder)

The build_installer and build_ISO expect to be run from a directory that points to the head of the stage (ex. /build/stage/8). This is usually a symlink to 8.1 or 8.1beta1. The script fully expands the symlink and pulls off the last element to figure out what version we are going to build.

VER=10
VERSION=$VER.0.alpha2
VERPREV=$VER.0.alpha1

mkdir -p /build/smeserver/stage/$VERSION/{SRPMS,x86_64}
cd /build/smeserver/stage/
rm -f $VER
ln -s $VERSION $VER

cd  /build/smeserver/stage/extra/images/$VER/product/

make any needed modification in the product image. then create the file and put it in place

find . | cpio -c -o | gzip -9cv > ../product.img; cd ..
cp product.img /build/smeserver/stage/$VER/images/

Local Only

mkdir -p /build/smeserver/stage/extra/isolinux/

Create cdrom content

When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.

	Warning:
	Note that this would remove everything else out of the target directory, eg translations.

Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/

rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme10/ /build/smeserver/stage/10/x86_64/
rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme10/isolinux/ /build/smeserver/stage/extra/isolinux/10/

OR Refresh cdrom content

If only refreshing the files, then remove the delete.

rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme10/ /build/smeserver/stage/10/x86_64/
rsync -a --exclude 'CVS/' ~/cdrom.image/sme10/isolinux/ /build/smeserver/stage/extra/isolinux/10/

Create the remaining folders

	Note:
	Note: images and isolinux do not need to be created as they are removed anyway in build_installer

x86_64

cd /build/smeserver/stage/10/x86_64
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/10/x86_64/repodata/
mkdir -p /build/smeserver/stage/10/x86_64/updates/po
mkdir -p /build/smeserver/stage/10/x86_64/updates/textw
mkdir -p /build/smeserver/stage/10/x86_64/updates/iw
mkdir -p /build/smeserver/stage/10/x86_64/updates/storage/devicelibs

other directories

mkdir -p /build/smeserver/stage/iso/10/x86_64
mkdir -p /build/smeserver/stage/iso/10/source

Copy anaconda translations

Anaconda translations need copying to /updates/po from ~/anaconda-po

cp -R ~/anaconda-po/translate.koozali.org/anaconda/sme10/* /build/smeserver/stage/10/i386/updates/po/

Copy anaconda content

See ~/cdrom.image/sme10/updates/README.txt for details of what to copy. The patched anaconda files from ~/smeserver/rpms/anaconda/sme10 will go to updates directory. The exact directory depends on the anaconda version, eg ~/smeserver/rpms/anaconda/sme10/anaconda-21.48.22.56

cd ~/smeserver/rpms/anaconda/sme10/
make prep
cd ~/smeserver/rpms/anaconda/sme10/anaconda-21.48.22.56

this does not work and need to be updated.

cp -R storage /build/smeserver/stage/10/x86_64/updates/
cp -R textw/*.py /build/smeserver/stage/10/x86_64/updates/textw
cp -R iw/*.py /build/smeserver/stage/10/x86_64/updates/iw
cp *.py /build/smeserver/stage/10/x86_64/updates
cp lang-table /build/smeserver/stage/10/x86_64/updates

Local Only: Hack: Permission Fixup

This is only needed on local server if permissions are incorrect. It should not be needed.

Local Only

sudo chmod -R a+w /build/smeserver/stage

Local Only: Create stage content

	Note:
	Note: This step occurs every hours on the buildsys.

Local Only

/build/smeserver/stage/bin/update_repos

Prepare SME Server 9

Local Only: Update local mirror as seen by the chroot

Local Only

rsync -av --delete --progress ~/smeserver/releases/9/ /build/smeserver/repo/9

Create release specific folder and link (or rename the folder)

mkdir -p /build/smeserver/stage/9.2/{SRPMS,i386,x86_64}
cd /build/smeserver/stage/
rm -f 9
ln -s 9.2 9

Local Only

mkdir -p /build/smeserver/stage/extra/isolinux/

Create cdrom content

When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.

	Warning:
	Note that this would remove everything else out of the target directory, eg translations.

Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/

rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64/
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386/
rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9/

OR Refresh cdrom content

If only refreshing the files, then remove the delete.

rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64/
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386/
rsync -a --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9/

Create the remaining folders

	Note:
	Note: images and isolinux do not need to be created as they are removed anyway in build_installer

i386

cd /build/smeserver/stage/9/i386
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/9/i386/repodata/
mkdir -p /build/smeserver/stage/9/i386/updates/po
mkdir -p /build/smeserver/stage/9/i386/updates/textw
mkdir -p /build/smeserver/stage/9/i386/updates/iw
mkdir -p /build/smeserver/stage/9/i386/updates/storage/devicelibs

x86_64

cd /build/smeserver/stage/9/x86_64
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/9/x86_64/repodata/
mkdir -p /build/smeserver/stage/9/x86_64/updates/po
mkdir -p /build/smeserver/stage/9/x86_64/updates/textw
mkdir -p /build/smeserver/stage/9/x86_64/updates/iw
mkdir -p /build/smeserver/stage/9/x86_64/updates/storage/devicelibs

other directories

mkdir -p /build/smeserver/stage/iso/9/i386
mkdir -p /build/smeserver/stage/iso/9/x86_64
mkdir -p /build/smeserver/stage/iso/9/source

Copy anaconda translations

Anaconda translations need copying to /updates/po from ~/anaconda-po

cp -R ~/anaconda-po/translate.koozali.org/anaconda/sme9/* /build/smeserver/stage/9/i386/updates/po/
cp -R ~/anaconda-po/translate.koozali.org/anaconda/sme9/* /build/smeserver/stage/9/x86_64/updates/po/

Copy anaconda content

See ~/cdrom.image/sme9/updates/README.txt for details of what to copy. The patched anaconda files from ~/smecore/rpms/anaconda/sme9 will go to updates directory. The exact directory depends on the anaconda version, eg ~/smecore/rpms/anaconda/sme9/anaconda-13.21.254

cd ~/smecore/rpms/anaconda/sme9/
make prep
cd ~/smecore/rpms/anaconda/sme9/anaconda-13.21.254
cp -R storage /build/smeserver/stage/9/i386/updates/
cp -R textw/*.py /build/smeserver/stage/9/i386/updates/textw
cp -R iw/*.py /build/smeserver/stage/9/i386/updates/iw
cp *.py /build/smeserver/stage/9/i386/updates
cp lang-table /build/smeserver/stage/9/i386/updates

cp -R storage /build/smeserver/stage/9/x86_64/updates/
cp -R textw/*.py /build/smeserver/stage/9/x86_64/updates/textw
cp -R iw/*.py /build/smeserver/stage/9/x86_64/updates/iw
cp *.py /build/smeserver/stage/9/x86_64/updates
cp lang-table /build/smeserver/stage/9/x86_64/updates

Local Only: Hack: Permission Fixup

This is only needed on local server if permissions are incorrect. It should not be needed.

Local Only

sudo chmod -R a+w /build/smeserver/stage

Local Only: Create stage content

	Note:
	Note: This step occurs every hours on the buildsys.

Local Only

/build/smeserver/stage/bin/update_repos

Prepare SME Server 8

Local Only: Update local mirror as seen by the chroot

rsync -av --delete --progress ~/smeserver/releases/8/ /build/smeserver/repo/8/

Create release specific folder and link (or rename folder)

mkdir -p /build/smeserver/stage/8.2/{SRPMS,i386,x86_64}
cd /build/smeserver/stage/
rm 8
ln -s 8.2 8

mkdir -p /build/smeserver/stage/extra/isolinux/

Create cdrom content

When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.

	Warning:
	Note that this would remove everything else out of the target directory, eg translations.

Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/

rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64/
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386/
rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8/

OR Refresh cdrom content

If only refreshing the files, then remove the delete.

rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64/
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386/
rsync -a --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8/

Create the remaining folders

	Note:
	Note: images and isolinux do not need to be created as they are removed anyway in build_installer

i386

cd /build/smeserver/stage/8/i386
ln -s SME Packages
mkdir -p /build/smeserver/stage/8/i386/repodata
mkdir -p /build/smeserver/stage/8/i386/updates/po
cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/i386/repodata/

x86_64

cd /build/smeserver/stage/8/x86_64
ln -s SME Packages
mkdir -p /build/smeserver/stage/8/x86_64/repodata
mkdir -p /build/smeserver/stage/8/x86_64/updates/po
cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/x86_64/repodata/

other directories

mkdir -p /build/smeserver/stage/iso/8/i386
mkdir -p /build/smeserver/stage/iso/8/x86_64
mkdir -p /build/smeserver/stage/iso/8/source

Copy anaconda translations

Anaconda translations need copying to /updates/po from ~/anaconda-po

cp -R ~/anaconda-po/translate.koozali.org/anaconda/sme8/* /build/smeserver/stage/8/i386/updates/po/
cp -R ~/anaconda-po/translate.koozali.org/anaconda/sme8/* /build/smeserver/stage/8/x86_64/updates/po/

Copy anaconda content

See ~/cdrom.image/sme8/updates/README.txt for details of what to copy. The patched anaconda files from ~/smecore/rpms/anaconda/sme8 will go to updates directory. The exact directory depends on the anaconda version, eg ~/smecore/rpms/anaconda/sme8/anaconda-11.1.2.263. One file (bootloaderInfo.py) also comes from booty. Finally lang-names needs to come from the actual anaconda RPM to updates which is a manual step.

cd ~/smecore/rpms/anaconda/sme8/
make prep
cd ~/smecore/rpms/anaconda/sme8/anaconda-11.1.2.263
cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/i386/updates/
cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/x86_64/updates/

cd textw
cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/i386/updates/
cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/x86_64/updates/

cd  ~/smecore/rpms/booty/sme8/
make prep
cp ~/smecore/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/i386/updates/
cp ~/smecore/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/x86_64/updates/

Local Only: Hack: Permission Fixup

This is only needed on local server if permissions are incorrect. It should not be needed.

Massage permissions before starting ISO building

Local Only

sudo chmod -R a+w /build/smeserver/stage

Local Only: Create stage content

	Note:
	Note: This step occurs every hours on the buildsys.

/build/smeserver/stage/bin/update_repos

Build & Release SME Server 10

VER=10
VERSION=$VER.0.alpha1
VERPREV=$VER.0.alpha0

You can find logs of the related scripts build_ISO and build_installer

If something goes wrong, please verify first

/build/smeserver/stage/$VER/build_installer.x86_64

and after if you have no errors

/build/smeserver/stage/$VER/build_ISO.x86_64

Build SME Server 10 64 bit ISO

rm -f /build/smeserver/stage/$VER/x86_64/SME
mock -r smeserver-$VER-x86_64-iso --init
mock -r smeserver-$VER-x86_64-iso --mount --cwd /build/smeserver/stage/$VER --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-$VER-x86_64-iso --mount --cwd /build/smeserver/stage/$VER --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/$VER/x86_64/
ln -s Packages SME
popd

local only

rm -f /build/smeserver/stage/$VER/x86_64/SME
mock -r smeserver-$VER-x86_64-iso --init
mock -r smeserver-$VER-x86_64-iso --mount --cwd /build/smeserver/stage/$VER --unpriv --chroot /build/smeserver/stage/bin/build_installer
sudo chmod -R a+w /build/smeserver/stage
mock -r smeserver-$VER-x86_64-iso --mount --cwd /build/smeserver/stage/$VER --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/$VER/x86_64/
ln -s Packages SME
popd

Build & Release SME Server 9

You can find logs of the related scripts build_ISO and build_installer

If something goes wrong, please verify first

/build/smeserver/stage/9/build_installer.i386

/build/smeserver/stage/9/build_installer.x86_64

and after if you have no errors

/build/smeserver/stage/9/build_ISO.i386

/build/smeserver/stage/9/build_ISO.x86_64

Build SME Server 9 32 bit ISO

rm -f /build/smeserver/stage/9/i386/SME
mock -r smeserver-9-i386-iso --init
mock -r smeserver-9-i386-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-9-i386-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/9/i386/
ln -s Packages SME
popd

local only

rm -f /build/smeserver/stage/9/i386/SME
mock -r smeserver-9-i386-iso --init
mock -r smeserver-9-i386-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
sudo chmod -R a+w /build/smeserver/stage
mock -r smeserver-9-i386-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/9/i386/
ln -s Packages SME
popd

Build SME Server 9 64 bit ISO

rm -f /build/smeserver/stage/9/x86_64/SME
mock -r smeserver-9-x86_64-iso --init
mock -r smeserver-9-x86_64-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-9-x86_64-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/9/x86_64/
ln -s Packages SME
popd

local only

rm -f /build/smeserver/stage/9/x86_64/SME
mock -r smeserver-9-x86_64-iso --init
mock -r smeserver-9-x86_64-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
sudo chmod -R a+w /build/smeserver/stage
mock -r smeserver-9-x86_64-iso --mount --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
pushd /build/smeserver/stage/9/x86_64/
ln -s Packages SME
popd

Build & Release SME Server 8

You can find logs of the related scripts build_ISO and build_installer

If something goes wrong, please verify first

/build/smeserver/stage/8/build_installer.i386

/build/smeserver/stage/8/build_installer.x86_64

and after if you have no errors

/build/smeserver/stage/8/build_ISO.i386

/build/smeserver/stage/8/build_ISO.x86_64

Build SME Server 8 32 bit ISO

mock -r smeserver-8-i386-iso --init
mock -r smeserver-8-i386-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-8-i386-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

local only

mock -r smeserver-8-i386-iso --init
mock -r smeserver-8-i386-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
sudo chmod -R a+w /build/smeserver/stage
mock -r smeserver-8-i386-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

Build SME Server 8 64 bit ISO

mock -r smeserver-8-x86_64-iso --init
mock -r smeserver-8-x86_64-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-8-x86_64-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

local only

mock -r smeserver-8-x86_64-iso --init
mock -r smeserver-8-x86_64-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
sudo chmod -R a+w /build/smeserver/stage
mock -r smeserver-8-x86_64-iso --mount --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

Where are My F... ISO

ls /build/smeserver/stage/iso/8/
i386  source  x86_64

and

ls /build/smeserver/stage/iso/9/
i386  source  x86_64

ls /build/smeserver/stage/iso/10/
source  x86_64

Developer Access Needed

From this point a developer access to the buildsys and the main repo is Needed, sorry guys

SME10

VER=10
VERSION=$VER.0.alpha1
VERPREV=$VER.0.alpha0

Prepare directory structure for SME10 release

SME 10.x test releases will go to /build/smeserver/repo/testing/9. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘10’ link will not be pointing to the new directory.

mkdir -p /build/smeserver/repo/testing/$VERSION/iso/source/
mkdir -p /build/smeserver/repo/testing/$VERSION/iso/x86_64/
mkdir -p /build/smeserver/repo/testing/$VERSION/smeos/x86_64/

Copy SME Server 10 64 bit Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/$VER/x86_64/smeserver-$VERSION-x86_64.jigdo /build/smeserver/repo/testing/$VERSION/iso/x86_64/
/bin/cp -a /build/smeserver/stage/iso/$VER/x86_64/smeserver-$VERSION-x86_64.template /build/smeserver/repo/testing/$VERSION/iso/x86_64/
rsync -a --delete /build/smeserver/stage/$VER/x86_64/ /build/smeserver/repo/testing/$VERSION/smeos/x86_64/

Copy SME Server 10 SRPMS Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/$VER/source/smeserver-$VERSION-SRPMS.jigdo /build/smeserver/repo/testing/$VERSION/iso/source/
/bin/cp -a /build/smeserver/stage/iso/$VER/source/smeserver-$VERSION-SRPMS.template /build/smeserver/repo/testing/$VERSION/iso/source/
rsync -a --delete /build/smeserver/stage/$VER/SRPMS/ /build/smeserver/repo/testing/$VERSION/smeos/SRPMS

Copy SME Server 10 ISOs to mirror

rsync -a --delete /build/smeserver/stage/iso/$VER/ /build/smeserver/repo/testing/$VERSION/iso

Hard links

As an example how to create the hard links:

cp -lr /build/smeserver/repo/testing/$VERPREV/smeos/* /build/smeserver/repo/testing/$VERSION/smeos/
mkdir -p /build/smeserver/repo/testing/$VERSION/smeaddons/
mkdir -p /build/smeserver/repo/testing/$VERSION/smecontribs/
mkdir -p /build/smeserver/repo/testing/$VERSION/smedev/
mkdir -p /build/smeserver/repo/testing/$VERSION/smeextras/
mkdir -p /build/smeserver/repo/testing/$VERSION/smetest/
mkdir -p /build/smeserver/repo/testing/$VERSION/smeupdates/
mkdir -p /build/smeserver/repo/testing/$VERSION/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/$VERPREV/smecontribs/* /build/smeserver/repo/testing/$VERSION/smecontribs/
cp -lr /build/smeserver/repo/testing/$VERPREV/smedev/* /build/smeserver/repo/testing/$VERSION/smedev/
cp -lr /build/smeserver/repo/testing/$VERPREV/smeextras/* /build/smeserver/repo/testing/$VERSION/smeextras/
cp -lr /build/smeserver/repo/testing/$VERPREV/smetest/* /build/smeserver/repo/testing/$VERSION/smetest/
cp -lr /build/smeserver/repo/testing/$VERPREV/smeupdates/* /build/smeserver/repo/testing/$VERSION/smeupdates/
cp -lr /build/smeserver/repo/testing/$VERPREV/smeupdates-testing/* /build/smeserver/repo/testing/$VERSION/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/$VERPREV/smeaddons/* /build/smeserver/repo/testing/$VERSION/smeaddons/

SME9

Prepare directory structure for SME9 release

SME 9.x test releases will go to /build/smeserver/repo/testing/9. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘9’ link will not be pointing to the new directory.

mkdir -p /build/smeserver/repo/testing/9.0/iso/i386/
mkdir -p /build/smeserver/repo/testing/9.0/iso/source/
mkdir -p /build/smeserver/repo/testing/9.0/iso/x86_64/
mkdir -p /build/smeserver/repo/testing/9.0/smeos/x86_64/
mkdir -p /build/smeserver/repo/testing/9.0/smeos/i386/

Copy SME Server 9 64 bit Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.jigdo /build/smeserver/repo/testing/9.0/iso/x86_64/
/bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.template /build/smeserver/repo/testing/9.0/iso/x86_64/
rsync -a --delete /build/smeserver/stage/9/x86_64/ /build/smeserver/repo/testing/9.0/smeos/x86_64/

Copy SME Server 9 32 bit Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.jigdo /build/smeserver/repo/testing/9.0/iso/i386/
/bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.template /build/smeserver/repo/testing/9.0/iso/i386/
rsync -a --delete /build/smeserver/stage/9/i386/ /build/smeserver/repo/testing/9.0/smeos/i386

Copy SME Server 9 SRPMS Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.jigdo /build/smeserver/repo/testing/9.0/iso/source/
/bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.template /build/smeserver/repo/testing/9.0/iso/source/
rsync -a --delete /build/smeserver/stage/9/SRPMS/ /build/smeserver/repo/testing/9.0/smeos/SRPMS

Copy SME Server 9 ISOs to mirror

rsync -a --delete /build/smeserver/stage/iso/9/ /build/smeserver/repo/testing/9.0/iso

Hard links

As an example how to create the hard links:

cp -lr /build/smeserver/repo/testing/9.0rc1/smeos/* /build/smeserver/repo/testing/9.0/smeos/
mkdir -p /build/smeserver/repo/testing/9.0/smeaddons/
mkdir -p /build/smeserver/repo/testing/9.0/smecontribs/
mkdir -p /build/smeserver/repo/testing/9.0/smedev/
mkdir -p /build/smeserver/repo/testing/9.0/smeextras/
mkdir -p /build/smeserver/repo/testing/9.0/smetest/
mkdir -p /build/smeserver/repo/testing/9.0/smeupdates/
mkdir -p /build/smeserver/repo/testing/9.0/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/9.0rc1/smecontribs/* /build/smeserver/repo/testing/9.0/smecontribs/
cp -lr /build/smeserver/repo/testing/9.0rc1/smedev/* /build/smeserver/repo/testing/9.0/smedev/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeextras/* /build/smeserver/repo/testing/9.0/smeextras/
cp -lr /build/smeserver/repo/testing/9.0rc1/smetest/* /build/smeserver/repo/testing/9.0/smetest/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates/* /build/smeserver/repo/testing/9.0/smeupdates/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates-testing/* /build/smeserver/repo/testing/9.0/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeaddons/* /build/smeserver/repo/testing/9.0/smeaddons/

SME8

Prepare directory structure for SME8 release

SME 8.x test releases will go to /build/smeserver/repo/testing/8. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘8’ link will not be pointing to the new directory.

mkdir -p /build/smeserver/repo/testing/8.1/iso/i386/
mkdir -p /build/smeserver/repo/testing/8.1/iso/source/
mkdir -p /build/smeserver/repo/testing/8.1/iso/x86_64/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/i386/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/SRPMS/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/x86_64/

Copy SME Server 8 64 bit Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.jigdo /build/smeserver/repo/testing/8.1/iso/x86_64/
/bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.template /build/smeserver/repo/testing/8.1/iso/x86_64/
rsync -a --delete /build/smeserver/stage/8/x86_64/ /build/smeserver/repo/testing/8.1/smeos/x86_64/

Copy SME Server 8 32 bit Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.jigdo /build/smeserver/repo/testing/8.1/iso/i386/
/bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.template /build/smeserver/repo/testing/8.1/iso/i386/
rsync -a --delete /build/smeserver/stage/8/i386/ /build/smeserver/repo/testing/8.1/smeos/i386

Copy SME Server 8 SRPMS Jigdo to mirror

/bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.jigdo /build/smeserver/repo/testing/8.1/iso/source/
/bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.template /build/smeserver/repo/testing/8.1/iso/source/
rsync -a --delete /build/smeserver/stage/8/SRPMS/ /build/smeserver/repo/testing/8.1/smeos/SRPMS

Copy SME Server 8 ISOs to mirror

rsync -a --delete /build/smeserver/stage/iso/8/ /build/smeserver/repo/testing/8.1/iso

@@ Line 323: / Line 323: @@
   ln -s $VERSION $VER
-  cd  /build/smeserver/stage/extra/images/10/product/
+  cd  /build/smeserver/stage/extra/images/$VER/product/
 make any needed modification in the product image. then create the file and put it in place
   find . | cpio -c -o | gzip -9cv > ../product.img; cd ..
-  cp product.img /build/smeserver/stage/10.0.alpha2/images/
+  cp product.img /build/smeserver/stage/$VER/images/
 '''Local Only'''

Difference between revisions of "Build an Official ISO"

Revision as of 11:25, 2 August 2016

Introduction

Folder structure, both local and buildsys

Local Only: mock requirement

Local Only: mock plugin : iso_prepare

Local Only: update_repos

Local Only: Create the repo folder and the binary program folder

Local Only: Mock Special Settings

Local Only: Patch a Mock Plugin

Preparation for a new ISO release

Generate a new Changelogs

The Smeserver version name

RPM dependencies

Examples of dependencies

Rpm update cycle

What are smeos/smeextras

How get a Fully updated ISO

RPM QA verification

Server-Manager translations

Anaconda translations

Check if installer needs re-building

Update cdrom.image content

Check Anaconda version

Bump smeserver-release if needed

Bump version in the splash screen

Remove the 'Beta warning' in Anaconda

For SME10

For SME9

For SME8

Prepare SME Server 10

Local Only: Update local mirror as seen by the chroot

Create release specific folder and link (or rename the folder)

Create cdrom content

OR Refresh cdrom content

Create the remaining folders

Copy anaconda translations

Copy anaconda content

Local Only: Hack: Permission Fixup

Local Only: Create stage content

Prepare SME Server 9

Local Only: Update local mirror as seen by the chroot

Create release specific folder and link (or rename the folder)

Create cdrom content

OR Refresh cdrom content

Create the remaining folders

Copy anaconda translations

Copy anaconda content

Local Only: Hack: Permission Fixup

Local Only: Create stage content

Prepare SME Server 8

Local Only: Update local mirror as seen by the chroot

Create release specific folder and link (or rename folder)

Create cdrom content

OR Refresh cdrom content

Create the remaining folders

Copy anaconda translations

Copy anaconda content

Local Only: Hack: Permission Fixup

Local Only: Create stage content

Build & Release SME Server 10

Build SME Server 10 64 bit ISO

local only

Build & Release SME Server 9

Build SME Server 9 32 bit ISO

local only

Build SME Server 9 64 bit ISO

local only

Build & Release SME Server 8

Build SME Server 8 32 bit ISO

local only

Build SME Server 8 64 bit ISO

local only

Where are My F... ISO

Developer Access Needed

SME10

Prepare directory structure for SME10 release

Copy SME Server 10 64 bit Jigdo to mirror

Copy SME Server 10 SRPMS Jigdo to mirror

Copy SME Server 10 ISOs to mirror