Difference between revisions of "Build an Official ISO"
| Line 103: | Line 103: | ||
| − | ====Anaconda version==== | + | ====Check Anaconda version==== |
Check which anaconda was used upstream, (eg http://vault.centos.org/5.11/os/SRPMS/ & http://vault.centos.org/6.6/os/Source/SPackages/) and is it newer than the one in CVS. If it is then anaconda needs to be updated and patched. | | Check which anaconda was used upstream, (eg http://vault.centos.org/5.11/os/SRPMS/ & http://vault.centos.org/6.6/os/Source/SPackages/) and is it newer than the one in CVS. If it is then anaconda needs to be updated and patched. | | ||
The latest Anaconda also needs to be in smeextras. | The latest Anaconda also needs to be in smeextras. | ||
Revision as of 16:27, 2 January 2015
Introduction
These Notes have been kindly shared by Ian Wells, thanks to him.
discussion on the Iso build
bugzilla:7675
build_installer and build_ISO are meant to be run in a mock chroot with the stage tree nfs mounted into it. The build box can be the same, eg COS6 x64 for all ISOs, it is only the mock image that needs to match the ISO, and this is magically handled by the mock configs. You also need to have access to the extras repo as many of the packages needed to build the installer and ISO are contained in there. This repo is to house the needed packages that aren't going to be included in the ISO. For clarification read http://bugs.contribs.org/show_bug.cgi?id=7675#c37
The os, updates, and updates-testing repos are included so we can stage the packages we are going to release without actually having to release them. Normally the updates-testing repo is used for verification (testing is for developer testing). During the ISO release schedule (usually very short) I hold moving packages to updates-testing until I have the ISO built.
The build_installer and build_ISO expect to be run from a directory that points to the head of the stage (ex. /build/stage/8). This is usually a symlink to 8.1 or 8.1beta1. The script fully expands the symlink and pulls off the last element to figure out what version we are going to build.
Folder structure, both local and buildsys
This document presumes that
~/builds_bin is builds_bin from CVS ~/cdrom.image is cdrom.image from CVS ~/anacondo-po contains the anaconda translations ~/smeserver contains a local mirror ~/rpms is rpms from CVS
Local Only: mock plugin : iso_prepare
You have to install a new plugin for mock : Iso_prepare
copy the content of http://bugs.contribs.org/attachment.cgi?id=4059 and paste it in
sudo vim /usr/lib/python2.6/site-packages/mockbuild/plugins/iso_prepare.py
Local Only: update_repos
update_repos relies on RPM2 which can be found as an RPM.
yum install perl-RPM2 --enablerepo=epel
update_repos (/build/smeserver/stage/bin/update_repos) is used to keep the repos in sync and to create the ‘stage’ from which the ISO is built. On the local build system update_repos can be run manually before building an ISO, whereas it runs every two hours on the buildsys.
The buildsys has two main folders that are relevant for building an ISO.
repo => '/build/smeserver/repo/ ', eg '/build/smeserver/repo/testing/9/' stage => '/build/smeserver/stage/', eg '/build/smeserver/stage/9/'
- repo is the folder that replicates to the mirrors. Be careful of all changes here.
- stage is the working directory. Note that update_repos expects the folder structure to be in place.
It also needs all GPG keys for every package that it sees, it can take a while to find them all.
Local Only: Create the repo folder and the binary program folder
Local Only
mkdir -p /build/smeserver/repo mkdir -p /build/smeserver/stage/bin
The content of builds_bin from CVS needs to be copied to /build/smeserver/stage/bin
Eg
/bin/cd ~/builds_bin /usr/bin/cvs update -dPA /bin/cp ~/builds_bin/* /build/smeserver/stage/bin
Local Only: Mock Special Settings
- These are the changes that you need to made if you do not have the nfs mounts on your local machine
config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/export/build', '/build', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/mirrors', '/mirrors', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
with
#config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/export/build', '/build', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
#config_opts['plugin_conf']['mount_opts']['dirs'].append(('storage:/mirrors', '/mirrors', 'nfs', 'defaults,noatime,nodiratime,nosuid'))
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/build', '/build' ))
Eg:
vim /etc/mock/smeserver-9-x86_64-iso.cfg
Local Only: Patch a Mock Plugin
There is also an update to /usr/lib/python2.6/site-packages/mockbuild/mounts.py that needs to happen so loop devices get mounted correctly. Check /var/lib/mock/smeserver-VERSION-ARCH/result/root.log after mock -r smeserver-VERSION-ARCH-iso --init (eg mock -r smeserver-8-i386-iso --init). The mounting of the /dev/loopX should pass (return code 0). If it fails (return code 32) then the patch below is needed on your build host. see http://bugs.contribs.org/show_bug.cgi?id=7675#c61
vim /usr/lib/python2.6/site-packages/mockbuild/mounts.py
@@ -83,6 +83,9 @@
decorate(traceLog())
def mount(self):
+ if not os.path.isdir(self.srcpath) and not os.path.isfile(self.srcpath):
+ mockbuild.util.touch(self.bindpath)
+
if not self.mounted:
cmd = ['/bin/mount', '-n',
'--bind', self.srcpath, self.bindpath ]
Preparation for a new ISO release
Server-Manager translations
Check translations from http://translate.contribs.org/patches/ if the filesize is >0 then need to patch smeserver-locale (to sme8 & sme9) and produce new rpms that will must be pushed to smeupdate-testing
Anaconda translations
Check installer translations from http://translate.contribs.org/anaconda/ I have no idea how to tell if there are updates get the latest files,
wget --mirror --no-parent -w 1 -A.mo -P ~/anacondo-po http://translate.contribs.org/anaconda/
Check if installer needs re-building
If either anaconda has changed OR there is new content in cdrom.image (apart from the release note) then the installer needs to be rebuilt (build_installer). Otherwise just the ISO needs to be built (build_ISO).
Update cdrom.image content
Update cdrom.image to the latest in CVS
cd ~/cdrom.image cvs update -dPA
Check Anaconda version
Check which anaconda was used upstream, (eg http://vault.centos.org/5.11/os/SRPMS/ & http://vault.centos.org/6.6/os/Source/SPackages/) and is it newer than the one in CVS. If it is then anaconda needs to be updated and patched. | The latest Anaconda also needs to be in smeextras.
SME Server 9
Local Only: Update local mirror as seen by the chroot
Local Only
rsync -av --delete --progress ~/smeserver/releases/9/ /build/smeserver/repo/9
Create release specific folder and link
mkdir -p /build/smeserver/stage/9.1/SRPMS cd /build/smeserver/stage/ ln -s 9.1/ 9
Local Only
mkdir -p /build/smeserver/stage/extra/isolinux/
Create cdrom content
When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.
Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64 rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386 rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9
OR Refresh cdrom content
If only refreshing the files, then remove the delete.
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/x86_64 rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme9/ /build/smeserver/stage/9/i386 rsync -a --exclude 'CVS/' ~/cdrom.image/sme9/isolinux/ /build/smeserver/stage/extra/isolinux/9
Create the remaining folders
- i386
cd /build/smeserver/stage/9/i386 ln -s Packages/ SME mkdir -p /build/smeserver/stage/9/i386/repodata/ mkdir -p /build/smeserver/stage/9/i386/updates/po mkdir -p /build/smeserver/stage/9/i386/updates/textw mkdir -p /build/smeserver/stage/9/i386/updates/iw
mkdir -p /build/smeserver/stage/9/i386/updates/storage/devicelibs
- x86_64
cd /build/smeserver/stage/9/x86_64 ln -s Packages/ SME mkdir -p /build/smeserver/stage/9/x86_64/repodata/ mkdir -p /build/smeserver/stage/9/x86_64/updates/po mkdir -p /build/smeserver/stage/9/x86_64/updates/textw mkdir -p /build/smeserver/stage/9/x86_64/updates/iw mkdir -p /build/smeserver/stage/9/x86_64/updates/storage/devicelibs
- other directories
mkdir -p /build/smeserver/stage/iso/9/i386 mkdir -p /build/smeserver/stage/iso/9/x86_64 mkdir -p /build/smeserver/stage/iso/9/source
Copy anaconda translations
Anaconda translations need copying to /updates/po from ~/anacondo-po
/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme9/* /build/smeserver/stage/9/i386/updates/po /bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme9/* /build/smeserver/stage/9/x86_64/updates/po
Copy anaconda content
See ~/cdrom.image/sme9/updates/README.txt for details of what to copy. The patched anaconda files from ~/rpms/anaconda/sme9 will go to updates directory. The exact directory depends on the anaconda version, eg ~/rpms/anaconda/sme9/anaconda-13.21.215
cd ~/rpms/anaconda/sme9/ make prep cd ~/rpms/anaconda/sme9/anaconda-13.21.215 /bin/cp -R storage /build/smeserver/stage/9/i386/updates/ /bin/cp -R textw/*.py /build/smeserver/stage/9/i386/updates/textw /bin/cp -R iw/*.py /build/smeserver/stage/9/i386/updates/iw /bin/cp *.py /build/smeserver/stage/9/i386/updates /bin/cp lang-table /build/smeserver/stage/9/i386/updates
/bin/cp -R storage /build/smeserver/stage/9/x86_64/updates/ /bin/cp -R textw/*.py /build/smeserver/stage/9/x86_64/updates/textw /bin/cp -R iw/*.py /build/smeserver/stage/9/x86_64/updates/iw /bin/cp *.py /build/smeserver/stage/9/x86_64/updates /bin/cp lang-table /build/smeserver/stage/9/x86_64/updates
Local Only: Hack: Permission Fixup
This is only needed on local server if permissions are incorrect. It should not be needed.
Local Only
sudo chmod -R a+w /build/smeserver/stage
Local Only: Create stage content
Local Only
/build/smeserver/stage/bin/update_repos
Prepare SME Server 8
Local Only: Update local mirror as seen by the chroot
rsync -av --delete --progress ~/smeserver/releases/8/ /build/smeserver/repo/8
Create release specific folder and link (or rename folder)
mkdir -p /build/smeserver/stage/8.1/SRPMS cd /build/smeserver/stage/ ln -s 8.1/ 8
mkdir -p /build/smeserver/stage/extra/isolinux/
Create cdrom content
When starting from fresh rsync can copy cdrom.image content, it also generates the folder tree.
Also note that the isolinux content goes into a temporary directory, /build/smeserver/stage/extra/isolinux/
rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64 rsync -a --delete --delete-excluded --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386 rsync -a --delete --delete-excluded --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8
OR Refresh cdrom content
If only refreshing the files, then remove the delete.
rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/x86_64 rsync -a --exclude 'CVS/' --exclude 'isolinux/' ~/cdrom.image/sme8/ /build/smeserver/stage/8/i386 rsync -a --exclude 'CVS/' ~/cdrom.image/sme8/isolinux/ /build/smeserver/stage/extra/isolinux/8
Create the remaining folders
- i386
cd /build/smeserver/stage/8/i386 ln -s Packages/ SME mkdir -p /build/smeserver/stage/8/i386/repodata/ mkdir -p /build/smeserver/stage/8/i386/updates/po /bin/cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/i386/repodata/
- x86_64
cd /build/smeserver/stage/8/x86_64 ln -s Packages/ SME mkdir -p /build/smeserver/stage/8/x86_64/repodata/ mkdir -p /build/smeserver/stage/8/x86_64/updates/po /bin/cp ~/cdrom.image/sme8/SME/base/comps.xml /build/smeserver/stage/8/x86_64/repodata/
- other directories
mkdir -p /build/smeserver/stage/iso/8/i386 mkdir -p /build/smeserver/stage/iso/8/x86_64 mkdir -p /build/smeserver/stage/iso/8/source
Copy anaconda translations
Anaconda translations need copying to /updates/po from ~/anacondo-po
/bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme8/* /build/smeserver/stage/8/i386/updates/po /bin/cp -R ~/anacondo-po/translate.contribs.org/anaconda/sme8/* /build/smeserver/stage/8/x86_64/updates/po
Copy anaconda content
See ~/cdrom.image/sme8/updates/README.txt for details of what to copy. The patched anaconda files from ~/rpms/anaconda/sme8 will go to updates directory. The exact directory depends on the anaconda version, eg ~/rpms/anaconda/sme8/anaconda-11.1.2.263. One file (bootloaderInfo.py) also comes from booty. Finally lang-names needs to come from the actual anaconda RPM to updates which is a manual step.
cd ~/rpms/anaconda/sme8/ make prep cd ~/rpms/anaconda/sme8/anaconda-11.1.2.263 /bin/cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/i386/updates/ /bin/cp anaconda bootloader.py findpackageset.py fsset.py instdata.py kickstart.py lang-table language.py packages.py partedUtils.py raid.py upgrade.py yuminstall.py /build/smeserver/stage/8/x86_64/updates/
cd textw /bin/cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/i386/updates/ /bin/cp confirm_text.py progress_text.py timezone_text.py upgrade_bootloader_text.py upgrade_text.py /build/smeserver/stage/8/x86_64/updates/
cd ~/rpms/booty/sme8/ make prep /bin/cp ~/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/i386/updates/ /bin/cp ~/rpms/booty/sme8/booty-0.80.6/bootloaderInfo.py /build/smeserver/stage/8/x86_64/updates/
Local Only: Hack: Permission Fixup
This is only needed on local server if permissions are incorrect. It should not be needed.
- Massage permissions before starting ISO building
Local Only
sudo chmod -R a+w /build/smeserver/stage
Local Only: Create stage content
/build/smeserver/stage/bin/update_repos
Build & Release SME Server 8
You can find logs of the related scripts build_ISO and build_installer
/build/smeserver/stage/8/build_ISO.XXX
and
/build/smeserver/stage/8/build_installer.XXX
Build SME Server 8 32 bit ISO
mock -r smeserver-8-i386-iso --init mock -r smeserver-8-i386-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer mock -r smeserver-8-i386-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
Build SME Server 8 64 bit ISO
mock -r smeserver-8-x86_64-iso --init mock -r smeserver-8-x86_64-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_installer mock -r smeserver-8-x86_64-iso --cwd /build/smeserver/stage/8 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
Prepare directory structure for release
SME 8.x test releases will go to /build/smeserver/repo/testing/8. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘8’ link will not be pointing to the new directory.
mkdir -p /build/smeserver/repo/testing/8.1/iso/i386/ mkdir -p /build/smeserver/repo/testing/8.1/iso/source/ mkdir -p /build/smeserver/repo/testing/8.1/iso/x86_64/ mkdir -p /build/smeserver/repo/testing/8.1/smeos/i386/ mkdir -p /build/smeserver/repo/testing/8.1/smeos/SRPMS/ mkdir -p /build/smeserver/repo/testing/8.1/smeos/x86_64/
Copy SME Server 8 64 bit Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.jigdo /build/smeserver/repo/testing/8.1/iso/x86_64/ /bin/cp -a /build/smeserver/stage/iso/8/x86_64/smeserver-8.1-x86_64.template /build/smeserver/repo/testing/8.1/iso/x86_64/ rsync -a --delete /build/smeserver/stage/8/x86_64/ /build/smeserver/repo/testing/8.1/smeos/x86_64
Copy SME Server 8 32 bit Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.jigdo /build/smeserver/repo/testing/8.1/iso/i386/ /bin/cp -a /build/smeserver/stage/iso/8/i386/smeserver-8.1-i386.template /build/smeserver/repo/testing/8.1/iso/i386/ rsync -a --delete /build/smeserver/stage/8/i386/ /build/smeserver/repo/testing/8.1/smeos/i386
Copy SME Server 8 SRPMS Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.jigdo /build/smeserver/repo/testing/8.1/iso/source/ /bin/cp -a /build/smeserver/stage/iso/8/source/smeserver-8.1-SRPMS.template /build/smeserver/repo/testing/8.1/iso/source/ rsync -a --delete /build/smeserver/stage/8/SRPMS/ /build/smeserver/repo/testing/8.1/smeos/SRPMS
Copy SME Server 8 ISOs to mirror
rsync -a --delete /build/smeserver/stage/iso/8/ /build/smeserver/repo/testing/8.1/iso
Build & Release SME Server 9
You can find logs of the related scripts build_ISO and build_installer
/build/smeserver/stage/9/build_ISO.XXX
and
/build/smeserver/stage/9/build_installer.XXX
Build SME Server 9 32 bit ISO
mock -r smeserver-9-i386-iso --init mock -r smeserver-9-i386-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer mock -r smeserver-9-i386-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
Build SME Server 9 64 bit ISO
mock -r smeserver-9-x86_64-iso --init mock -r smeserver-9-x86_64-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_installer mock -r smeserver-9-x86_64-iso --cwd /build/smeserver/stage/9 --unpriv --chroot /build/smeserver/stage/bin/build_ISO
Prepare directory structure for release
SME 9.x test releases will go to /build/smeserver/repo/testing/9. Check that the link points to the correct testing release. Note when uploading Jigdos the ‘9’ link will not be pointing to the new directory.
mkdir -p /build/smeserver/repo/testing/9.0/iso/i386/ mkdir -p /build/smeserver/repo/testing/9.0/iso/source/ mkdir -p /build/smeserver/repo/testing/9.0/iso/x86_64/ mkdir -p /build/smeserver/repo/testing/9.0/smeos/x86_64/ mkdir -p /build/smeserver/repo/testing/9.0/smeos/i386/
Copy SME Server 9 64 bit Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.jigdo /build/smeserver/repo/testing/9.0/iso/x86_64/ /bin/cp -a /build/smeserver/stage/iso/9/x86_64/smeserver-9.0-x86_64.template /build/smeserver/repo/testing/9.0/iso/x86_64/ rsync -a --delete /build/smeserver/stage/9/x86_64/ /build/smeserver/repo/testing/9.0/smeos/x86_64
Copy SME Server 9 32 bit Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.jigdo /build/smeserver/repo/testing/9.0/iso/i386/ /bin/cp -a /build/smeserver/stage/iso/9/i386/smeserver-9.0-i386.template /build/smeserver/repo/testing/9.0/iso/i386/ rsync -a --delete /build/smeserver/stage/9/i386/ /build/smeserver/repo/testing/9.0/smeos/i386
Copy SME Server 9 SRPMS Jigdo to mirror
/bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.jigdo /build/smeserver/repo/testing/9.0/iso/source/ /bin/cp -a /build/smeserver/stage/iso/9/source/smeserver-9.0-SRPMS.template /build/smeserver/repo/testing/9.0/iso/source/ rsync -a --delete /build/smeserver/stage/9/SRPMS/ /build/smeserver/repo/testing/9.0/smeos/SRPMS
Copy SME Server 9 ISOs to mirror
rsync -a --delete /build/smeserver/stage/iso/9/ /build/smeserver/repo/testing/9.0/iso
Hard links
As an example how to create the hard links:
cp -lr /build/smeserver/repo/testing/9.0rc1/smeos/* /build/smeserver/repo/testing/9.0/smeos/ mkdir -p /build/smeserver/repo/testing/9.0/smeaddons/ mkdir -p /build/smeserver/repo/testing/9.0/smecontribs/ mkdir -p /build/smeserver/repo/testing/9.0/smedev/ mkdir -p /build/smeserver/repo/testing/9.0/smeextras/ mkdir -p /build/smeserver/repo/testing/9.0/smetest/ mkdir -p /build/smeserver/repo/testing/9.0/smeupdates/ mkdir -p /build/smeserver/repo/testing/9.0/smeupdates-testing/ cp -lr /build/smeserver/repo/testing/9.0rc1/smecontribs/* /build/smeserver/repo/testing/9.0/smecontribs/ cp -lr /build/smeserver/repo/testing/9.0rc1/smedev/* /build/smeserver/repo/testing/9.0/smedev/ cp -lr /build/smeserver/repo/testing/9.0rc1/smeextras/* /build/smeserver/repo/testing/9.0/smeextras/ cp -lr /build/smeserver/repo/testing/9.0rc1/smetest/* /build/smeserver/repo/testing/9.0/smetest/ cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates/* /build/smeserver/repo/testing/9.0/smeupdates/ cp -lr /build/smeserver/repo/testing/9.0rc1/smeupdates-testing/* /build/smeserver/repo/testing/9.0/smeupdates-testing/ cp -lr /build/smeserver/repo/testing/9.0rc1/smeaddons/* /build/smeserver/repo/testing/9.0/smeaddons/
