Changes

Jump to navigation Jump to search
17,729 bytes added ,  16:30, 24 December 2014
Created page with "{{Level|Developer}} {{WIP box|}} ==Introduction== ====Folder structure, both local and buildsys==== This document presumes that ~/builds_bin is builds_bin from CVS ~/cd..."
{{Level|Developer}}

{{WIP box|}}

==Introduction==

====Folder structure, both local and buildsys====

This document presumes that
~/builds_bin is builds_bin from CVS
~/cdrom.image is cdrom.image from CVS
~/anacondo-po contains '''the anaconda translations'''
~/smeserver contains a local mirror
~/rpms is rpms from CVS

====update_repos====

update_repos (/build/smeserver/stage/bin/update_repos) is used to keep the repos in sync and to create the ‘stage’ from which the ISO is built. On the local build system update_repos can be run manually before building an ISO, whereas it runs every two hours on the buildsys.

The buildsys has two main folders that are relevant for building an ISO.
repo => '/build/smeserver/repo/ ', eg '/build/smeserver/repo/testing/9/'
stage => '/build/smeserver/stage/', eg '/build/smeserver/stage/9/'

* '''repo''' is the folder that replicates to the mirrors. Be careful of all changes here.
* '''stage''' is the working directory. Note that update_repos expects the folder structure to be in place.

update_repos relies on RPM2 which can be found as an RPM.
It also needs all GPG keys for every package that it sees, it can take a while to find them all.

Note that update_repos copies smeupdates-testing packages to stage but not from smetest.

===='''Local Only''': Create the repo folder and the binary program folder====
mkdir -p /build/smeserver/repo
mkdir -p /build/smeserver/stage/bin

The content of builds_bin from CVS needs to be copied to /build/smeserver/stage/bin

Eg
/bin/cp ~/builds_bin/* /build/smeserver/stage/bin


====Building the SME Server ISOs====
{{Note box|Note I needed to edit mock cfg file to include /build in the chroot due to my local setup.}}
There is also an update to /usr/lib/python2.6/site-packages/mockbuild/mounts.py

===Preparation for a new ISO release===

====Server-Manager translations====
Check translations from http://translate.contribs.org/patches/ if the filesize is >0 then need to patch smeserver-locale

====Anaconda translations====
Check installer translations from http://translate.contribs.org/anaconda/ I have no idea how to tell if there are updates get the latest files,

wget --mirror --no-parent -A.mo -P ~/anacondo-po http://translate.contribs.org/anaconda/

====Check if installer needs re-building====
If either anaconda has changed OR there is new content in cdrom.image (apart from the release note) then the installer needs to be rebuilt (build_installer). Otherwise just the ISO needs to be built (build_ISO).

====Update cdrom.image content====
Update cdrom.image to the latest in CVS

cd ~/cdrom.image
cvs update -dPA


====Anaconda version====
Check which anaconda was used upstream, eg http://vault.centos.org/5.10/os/SRPMS/ and is it newer than the one in CVS. If it is then anaconda needs to be updated and patched. |
The latest Anaconda also needs to be in smeextras.



===SME Server 9===

===='''Local Only''': Update local mirror as seen by the chroot====
rsync -av --delete ~/smeserver/releases/testing/9/ /build/smeserver/repo/testing/9



====Create release specific folder and link====
mkdir -p /build/smeserver/stage/9.0rc1/SRPMS
cd /build/smeserver/stage/
ln -s 9.0rc1/ 9

'''Local Only'''
mkdir -p /build/smeserver/stage/extra/isolinux/

====Create cdrom content====
When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.
{{Note box|Note that this would remove everything else out of the target directory, eg translations.}}
Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/

rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386
rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9


====OR Refresh cdrom content====
If only refreshing the files, then remove the delete.

rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386
rsync -a --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9


====Create the remaining folders====
{{Note box|Note: images and isolinux do not need to be created as they are removed anyway in build_installer}}

* i386
cd /build/smeserver/stage/9/i386
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/9/i386/repodata/
mkdir -p /build/smeserver/stage/9/i386/updates/po
mkdir -p /build/smeserver/stage/9/i386/updates/textw
mkdir -p /build/smeserver/stage/9/i386/updates/iw

mkdir -p /build/smeserver/stage/9/i386/updates/storage/devicelibs

* x86_64
cd /build/smeserver/stage/9/x86_64
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/9/x86_64/repodata/
mkdir -p /build/smeserver/stage/9/x86_64/updates/po
mkdir -p /build/smeserver/stage/9/x86_64/updates/textw
mkdir -p /build/smeserver/stage/9/x86_64/updates/iw
mkdir -p /build/smeserver/stage/9/x86_64/updates/storage/devicelibs

* other directories
mkdir -p /build/smeserver/stage/iso/9/i386
mkdir -p /build/smeserver/stage/iso/9/x86_64
mkdir -p /build/smeserver/stage/iso/9/source

====Copy anaconda translations====
Anaconda translations need copying to /updates/po from ~/anacondo-po
/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme9/* /build/smeserver/stage/9/i386/updates/po
/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme9/* /build/smeserver/stage/9/x86_64/updates/po

====Copy anaconda content====
See ~/cdrom.image/sme9/updates/README.txt for details of what to copy. The patched anaconda files from ~/rpms/anaconda/sme9 will go to updates directory. The exact directory depends on the anaconda version, eg ~/rpms/anaconda/sme9/anaconda-13.21.215

cd ~/rpms/anaconda/sme9/anaconda-13.21.215
/bin/cp -R storage /build/smeserver/stage/9/i386/updates/
/bin/cp -R textw/*.py /build/smeserver/stage/9/i386/updates/textw
/bin/cp -R iw/*.py /build/smeserver/stage/9/i386/updates/iw
/bin/cp *.py /build/smeserver/stage/9/i386/updates
/bin/cp lang-table /build/smeserver/stage/9/i386/updates

/bin/cp -R storage /build/smeserver/stage/9/x86_64/updates/
/bin/cp -R textw/*.py /build/smeserver/stage/9/x86_64/updates/textw
/bin/cp -R iw/*.py /build/smeserver/stage/9/x86_64/updates/iw
/bin/cp *.py /build/smeserver/stage/9/x86_64/updates
/bin/cp lang-table /build/smeserver/stage/9/x86_64/updates

===='''Local Only''': Hack: Permission Fixup====
This is only needed on local server if permissions are incorrect. It should not be needed.
sudo chmod -R a+w /build/smeserver/stage

===='''Local Only''': Create stage content====
{{Note box|Note: This step occurs every two hours on the buildsys.}}
/build/smeserver/stage/bin/update_repos

===Prepare SME Server 8===

===='''Local Only''': Update local mirror as seen by the chroot====
rsync -av --delete ~/smeserver/releases/8/ /build/smeserver/repo/8


====Create release specific folder and link (or rename folder)====
mkdir -p /build/smeserver/stage/8.1/SRPMS
cd /build/smeserver/stage/
ln -s 8.1/ 8

mkdir -p /build/smeserver/stage/extra/isolinux/


====Create cdrom content====
When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.
{{Note box|Note that this would remove everything else out of the target directory, eg translations.}}
Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/

rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386
rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8


====OR Refresh cdrom content====
If only refreshing the files, then remove the delete.

rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386
rsync -a --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8


====Create the remaining folders====
{{Note box|Note: images and isolinux do not need to be created as they are removed anyway in build_installer}}

* i386
cd /build/smeserver/stage/8/i386
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/8/i386/repodata/
mkdir -p /build/smeserver/stage/8/i386/updates/po
/bin/cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/i386/repodata/

* x86_64
cd /build/smeserver/stage/8/x86_64
ln -s Packages/ SME
mkdir -p /build/smeserver/stage/8/x86_64/repodata/
mkdir -p /build/smeserver/stage/8/x86_64/updates/po
/bin/cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/x86_64/repodata/

* other directories
mkdir -p /build/smeserver/stage/iso/8/i386
mkdir -p /build/smeserver/stage/iso/8/x86_64
mkdir -p /build/smeserver/stage/iso/8/source


====Copy anaconda translations====
Anaconda translations need copying to /updates/po from ~/anacondo-po

/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme8/* /build/smeserver/stage/8/i386/updates/po
/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme8/* /build/smeserver/stage/8/x86_64/updates/po


====Copy anaconda content====
See ~/cdrom.image/sme8/updates/README.txt for details of what to copy. The patched anaconda files from ~/rpms/anaconda/sme8 will go to updates directory. The exact directory depends on the anaconda version, eg ~/rpms/anaconda/sme8/anaconda-11.1.2.263. One file (bootloaderInfo.py) also comes from booty. '''Finally lang-names needs to come from the actual anaconda RPM to updates which is a manual step.'''

cd ~/rpms/anaconda/sme8/anaconda-11.1.2.263
/bin/cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/i386/updates/
/bin/cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/x86_64/updates/

cd textw
/bin/cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/i386/updates/
/bin/cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/x86_64/updates/

/bin/cp ~/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/i386/updates/
/bin/cp ~/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/x86_64/updates/

===='''Local Only''': Hack: Permission Fixup====
This is only needed on local server if permissions are incorrect. '''It should not be needed.'''
# Massage permissions before starting ISO building
sudo chmod -R a+w /build/smeserver/stage

===='''Local Only''': Create stage content====
{{Note box|Note: This step occurs every two hours on the buildsys.}}
/build/smeserver/stage/bin/update_repos

===Build & Release SME Server 8===

====Build SME Server 8 32 bit ISO====
mock -r smeserver-8-i386-iso --init
mock -r smeserver-8-i386-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-8-i386-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

====Build SME Server 8 64 bit ISO====
mock -r smeserver-8-x86_64-iso --init
mock -r smeserver-8-x86_64-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-8-x86_64-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

====Prepare directory structure for release====
SME 8.x test releases will go to /build/smeserver/repo/testing/8. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘8’ link will not be pointing to the new directory.
mkdir -p /build/smeserver/repo/testing/8.1/iso/i386/
mkdir -p /build/smeserver/repo/testing/8.1/iso/source/
mkdir -p /build/smeserver/repo/testing/8.1/iso/x86_64/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/i386/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/SRPMS/
mkdir -p /build/smeserver/repo/testing/8.1/smeos/x86_64/

====Copy SME Server 8 64 bit Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.jigdo /build/smeserver/repo/testing/8.1/iso/x86_64/
/bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.template /build/smeserver/repo/testing/8.1/iso/x86_64/
rsync -a --delete /build/smeserver/stage/8/x86_64/ /build/smeserver/repo/testing/8.1/smeos/x86_64

====Copy SME Server 8 32 bit Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.jigdo /build/smeserver/repo/testing/8.1/iso/i386/
/bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.template /build/smeserver/repo/testing/8.1/iso/i386/
rsync -a --delete /build/smeserver/stage/8/i386/ /build/smeserver/repo/testing/8.1/smeos/i386

====Copy SME Server 8 SRPMS Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.jigdo /build/smeserver/repo/testing/8.1/iso/source/
/bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.template /build/smeserver/repo/testing/8.1/iso/source/
rsync -a --delete /build/smeserver/stage/8/SRPMS/ /build/smeserver/repo/testing/8.1/smeos/SRPMS

====Copy SME Server 8 ISOs to mirror====
rsync -a --delete /build/smeserver/stage/iso/8/ /build/smeserver/repo/testing/8.1/iso
===Build & Release SME Server 9===

===Build SME Server 9 32 bit ISO===
mock -r smeserver-9-i386-iso --init
mock -r smeserver-9-i386-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-9-i386-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

====Build SME Server 9 64 bit ISO====
mock -r smeserver-9-x86_64-iso --init
mock -r smeserver-9-x86_64-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer
mock -r smeserver-9-x86_64-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO

====Prepare directory structure for release====
SME 9.x test releases will go to /build/smeserver/repo/testing/9. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘9’ link will not be pointing to the new directory.
mkdir -p /build/smeserver/repo/testing/9.0/iso/i386/
mkdir -p /build/smeserver/repo/testing/9.0/iso/source/
mkdir -p /build/smeserver/repo/testing/9.0/iso/x86_64/
mkdir -p /build/smeserver/repo/testing/9.0/smeos/x86_64/
mkdir -p /build/smeserver/repo/testing/9.0/smeos/i386/

====Copy SME Server 9 64 bit Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.jigdo /build/smeserver/repo/testing/9.0/iso/x86_64/
/bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.template /build/smeserver/repo/testing/9.0/iso/x86_64/
rsync -a --delete /build/smeserver/stage/9/x86_64/ /build/smeserver/repo/testing/9.0/smeos/x86_64

====Copy SME Server 9 32 bit Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.jigdo /build/smeserver/repo/testing/9.0/iso/i386/
/bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.template /build/smeserver/repo/testing/9.0/iso/i386/
rsync -a --delete /build/smeserver/stage/9/i386/ /build/smeserver/repo/testing/9.0/smeos/i386

====Copy SME Server 9 SRPMS Jigdo to mirror====
/bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.jigdo /build/smeserver/repo/testing/9.0/iso/source/
/bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.template /build/smeserver/repo/testing/9.0/iso/source/
rsync -a --delete /build/smeserver/stage/9/SRPMS/ /build/smeserver/repo/testing/9.0/smeos/SRPMS

====Copy SME Server 9 ISOs to mirror====
rsync -a --delete /build/smeserver/stage/iso/9/ /build/smeserver/repo/testing/9.0/iso



===Hard links===
As an example how to create the hard links:
cp -lr /build/smeserver/repo/testing/9.0rc1/smeos/* /build/smeserver/repo/testing/9.0/smeos/
mkdir -p /build/smeserver/repo/testing/9.0/smeaddons/
mkdir -p /build/smeserver/repo/testing/9.0/smecontribs/
mkdir -p /build/smeserver/repo/testing/9.0/smedev/
mkdir -p /build/smeserver/repo/testing/9.0/smeextras/
mkdir -p /build/smeserver/repo/testing/9.0/smetest/
mkdir -p /build/smeserver/repo/testing/9.0/smeupdates/
mkdir -p /build/smeserver/repo/testing/9.0/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/9.0rc1/smecontribs/* /build/smeserver/repo/testing/9.0/smecontribs/
cp -lr /build/smeserver/repo/testing/9.0rc1/smedev/* /build/smeserver/repo/testing/9.0/smedev/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeextras/* /build/smeserver/repo/testing/9.0/smeextras/
cp -lr /build/smeserver/repo/testing/9.0rc1/smetest/* /build/smeserver/repo/testing/9.0/smetest/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates/* /build/smeserver/repo/testing/9.0/smeupdates/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates-testing/* /build/smeserver/repo/testing/9.0/smeupdates-testing/
cp -lr /build/smeserver/repo/testing/9.0rc1/smeaddons/* /build/smeserver/repo/testing/9.0/smeaddons/

Navigation menu