RPM Tools
Rpm Tools
Introduction
In this howto we could let script-tools we need to build rpm
Stephdl
stephdl Stéphane de Labrusse AKA Stephdl
mockbuild
mock get a special syntax not always easy to recall, here a script that you need to record at /usr/bin/mockbuild (set it executable) https://github.com/stephdl/mockbuild
- mockbuild usage
mockbuild package.src.rpm mock_configuration
mock_configuration can be found in /etc/mocks
smeserver-9-x86_64-base smeserver-9-i386-base smeserver-8-x86_64-base smeserver-8-i386-base epel-6-x86_64 epel-6-i386 epel-5-x86_64 epel-5-i386
#!/bin/bash # who i'm ME=$(whoami) mkdir -p /home/$ME/exchange #test if the second argument is here if [[ -z $2 ]]; then echo "################################################" echo "you must give as argument the mock configuration" echo " smeserver-9-x86_64-base smeserver-9-i386-base" echo " smeserver-8-x86_64-base smeserver-8-i386-base" echo " epel-6-x86_64 epel-6-i386 " echo " epel-5-x86_64 epel-5-i386 " echo " mockbuild package.src.rpm mock_configuration " echo "################################################" exit fi mock -r $2 --rebuild $1 --resultdir=/home/$ME/exchange/$2 # we give the path where are files echo "" echo "==================================================================================================================" echo "RPMS are in /home/$ME/exchange/$2 of your build server" echo "" ls -1 /home/$ME/exchange/$2/*.rpm echo "==================================================================================================================" # and we sign rpm if needed, if not simply do 'enter' with keyboard echo "" echo "3.Sign your rpms" rpm --resign /home/$ME/exchange/$2/*.rpm 2>&1 > /dev/null
A the end I use rpm to sign my rpm, if you don't need it you can remove it.
git_ignore
git don't track empty directory that we may need for a rpm, here a script that you need to record at /usr/bin/git_ignore (set it executable) https://github.com/stephdl/git_ignore
usage in the git directory call the script and track all gew files by " git add . && git commit -m 'your-message' "
git_ignore
#in order to add a .gitignore file to track empty folder find * -type d -empty -exec touch {}/.gitignore \;
git_mockbuild
a git integration for mock, git and rpm https://github.com/stephdl/git_mockbuild
you can record this script in /usr/bin
cp git_mockbuild /usr/bin chmod 775 /usr/bin/git_mockbuild
you need to make a folder with the exact name of the rpm (eg smeserver-dhcpmanager). Inside of this folder you create your git repository and work on files and folder of your RPM. Be aware that GIT do not track empty repositories, you have to create a '.gitignore' (even with nothing inside) in each empty directory, if you want they will be incorporated in your GIT repository.
you can use this command line in case of many folders and subfolders
find * -type d -empty -exec touch {}/.gitignore \;
Once you want to make the RPM then simply use git_mockbuild without arguments in the root folder of your git repository, the architecture of build come from the spec file. But if needed you can choose another build arch. You have to choose one argument.
git_mockbuild or git_mockbuild sme8-386 sme8-64 sme9-386 sme9-64
by default if the build architecture is not specified in the spec file, it is the case of packages which are not 'noarch', the RPM will be make by mock with the same architecture of your build server. Therefore you have to specify which arch you want if you need another rpm architecture (eg i386 instead of x86_64)
The CentOS target version comes from the GIT branch you created in your GIT repository. For example if your branch is named sme9 then the build is done for el6, else the target CentOS version is el5.
For an automatic build for all architectures you should take considerations about Plague which is done for that.
Each time you execute the script, the GIT branch in use is tagged with the version of the RPM.
The result of build can be find in the 'exchange' folder of the user's home of that script
- usage
in the git directory of your rpm (you need to install git & mock installed)
git_mockbuild
#!/bin/bash ############################################################################################## ## a big thank to daniel berteaud from firewall-service who started in first to write this script ## the purpose of this script is to create a targ.gz from the GIT folder in order to build a rpm ## then after mock is used to make the rpm in a chroot ############################################################################################## # git_mockbuild [ MOCKCONF ] [ SME_Version ] # parameters are optional ME=$(whoami) # What is the package name # The package name is given by the name of the main folder PACKAGE=$(basename $(pwd)) # Informations needed from the spec file VERSION=$(rpm -q --qf "%{version}\n" --specfile $PACKAGE.spec | head -1) ARCH=$(rpm -q --qf "%{arch}\n" --specfile $PACKAGE.spec | head -1) RELEASE=$(rpm -q --qf "%{release}\n" --specfile $PACKAGE.spec | head -1) # Which git branch we are talking BRANCH=$(git branch | grep '*' | cut -d' ' -f2) # Find the centos version DIST=$2 if -z $DIST && "$BRANCH" =~ "sme9" ; then DIST="el6" elif [ -z $DIST ]; then DIST="el5" fi # Create a tmp folder in /tmp TMPDIR=$(mktemp -d) mkdir -p $TMPDIR/$PACKAGE-$VERSION/ # Tag GIT using the version in the spec file ##we do a hack to retrieve the release without %{?dist} RELEASETAG=${RELEASE/.el6/} TAG=$VERSION"-"$RELEASETAG"_"$DIST git tag -f $TAG 2>&1 > /dev/null # Create needed dirs on the build box mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,SOURCES} mkdir -p ~/exchange # Generate a tgz archive from the repo git archive --format=tar --prefix=$PACKAGE-$VERSION/ $BRANCH | tar xf - -C $TMPDIR # Generate ChangeLog git log --format=%H | git log --pretty --stdin --no-walk > $TMPDIR/$PACKAGE-$VERSION/CHANGELOG.git # Create the tar.gz archive tar czf $TMPDIR/$PACKAGE-$VERSION.tar.gz -C $TMPDIR $PACKAGE-$VERSION # cp the spec file and tar.gz cp $TMPDIR/$PACKAGE-$VERSION.tar.gz ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz cp $PACKAGE.spec ~/rpmbuild/SPECS/ # And build echo"" echo "0.Tips" echo "# git_mockbuild [ MOCKCONF ] [ SME_Server_version ]" echo 'you can choose another build architecture : git_mockbuild sme8-386 sme8-64 sme9-386 sme9-64 epel-6-x86_64 epel-6-i386 epel-5-x86_64 epel-5-i386' echo 'you can choose which SME_Server target version you want : sme8 sme9' echo "" echo "1.Build the src.rpm" echo "" cd ~/rpmbuild/SOURCES/ tar -xvf ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz 2>&1 > /dev/null rpmbuild -bs --nodeps ~/rpmbuild/SPECS/$PACKAGE.spec echo "" echo "2.Build with Mock" echo "" echo "==============================================================" echo "The build RPM architecture is $ARCH" echo "The target CentOS version is $DIST" if [ "$DIST" == "el5" ]; then if [ "$ARCH" == "noarch" ]; then MOCKCONF="smeserver-8-i386-base" elif [ "$ARCH" == "x86_64" ]; then MOCKCONF="smeserver-8-x86_64-base" elif [ "$ARCH" == "i386" ]; then MOCKCONF="smeserver-8-i386-base" fi; elif [ "$DIST" == "el6" ]; then if [ "$ARCH" == "noarch" ]; then MOCKCONF="smeserver-9-i386-base" elif [ "$ARCH" == "x86_64" ]; then MOCKCONF="smeserver-9-x86_64-base" elif [ "$ARCH" == "i386" ]; then MOCKCONF="smeserver-9-i386-base" fi; fi if ! [ -z "$1" ]; then if [ $1 = "sme8-386" ]; then MOCKCONF="smeserver-8-i386-base" elif [ $1 = "sme8-64" ]; then MOCKCONF="smeserver-8-x86_64-base" elif [ $1 = "sme9-386" ]; then MOCKCONF="smeserver-9-i386-base" elif [ $1 = "sme9-64" ]; then MOCKCONF="smeserver-9-x86_64-base" elif [ $1 = "epel-6-x86_64" ]; then MOCKCONF="epel-6-x86_64" elif [ $1 = "epel-6-i386" ]; then MOCKCONF="epel-6-i386" elif [ $1 = "epel-5-x86_64" ]; then MOCKCONF="epel-5-x86_64" elif [ $1 = "epel-5-i386" ]; then MOCKCONF="epel-5-i386" fi; fi echo "The mock chroot configuration used is $MOCKCONF" echo "==============================================================" echo "" mock -r $MOCKCONF --rebuild ~/rpmbuild/SRPMS/$PACKAGE-$VERSION-*.src.rpm --resultdir=/home/$ME/exchange/$PACKAGE-$VERSION # a bit of clean rm -f ~/rpmbuild/SPECS/$PACKAGE.spec ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz ~/rpmbuild/SRPMS/$PACKAGE-$VERSION*.src.rpm rm -rf $TMPDIR ~/$PACKAGE-$VERSION # we give the path where are files echo "" echo "==================================================================================================================" echo "RPMS are in /home/$ME/exchange/$PACKAGE-$VERSION of your build server" echo "" ls -1 ~/exchange/$PACKAGE-$VERSION/*.rpm echo "==================================================================================================================" # and we sign rpm if needed, if not simply do 'enter' with keyboard echo "" echo "3.Sign your rpms" rpm --resign ~/exchange/$PACKAGE-$VERSION/*.rpm 2>&1 > /dev/null
contrib_migration
a script to migrate contrib from sme8 CVS to your local git and to your remote Github account for sme9 https://github.com/stephdl/contrib_migration . You need to record at /usr/bin/contrib_migration (set it executable)
simply call the script with the name of the cvs directory. Of course you have to install cvs and retrieve the source code in an anonymous or with a developer access
contrib_migration smeserver-wordpress
If the directory is not existed in your git directory (~/git_work) in my server, then we launch the script. After that it creates automatically the new repository in github with your credentials (adjust 'stephdl' with your login account and replace 'xxxxxxxxxxxxxx' with your password) After that the script does several things
- create the local git repository
- create the remote github account
- build all patches and remove them
- move all perl files to the correct new path
- test if the new apache authentication is needed and where.
- add a .git_ignore in empty folders (needed by git)
- track all that stuff :)
- usage
you need to be outside of the cvs folder (eg in the 'rpms' folder)
contrib_migration smeserver-packagename
#test if the first argument is here if [[ -z $1 ]]; then echo "#################################################################################" echo "you must give as argument the folder name in cvs : contrib_migration folder_name" echo "#################################################################################" exit fi if [[ ! -d ~/git_work/$1 ]]; then rm -rf $1/contribs8 cvs co $1/contribs8 cd $1/contribs8 make prep NameGitHub=$1 curl -u 'stephdl:xxxxxxxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" , "description": "A contrib for SME9", "homepage": "http://wiki.contribs.org", "private": false, "has_issues": true, "has_wiki": true, "has_downloads": true}' mkdir -p ~/git_work/$1 cp -R smeserver-*/* ~/git_work/$1 cp $1*.spec ~/git_work/$1 cd ~/git_work/$1 if [ -e root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/ ]; then mkdir -p root/usr/share/perl5/vendor_perl/esmith/FormMagick/Panel/ mv root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/* root/usr/share/perl5/vendor_perl/esmith/FormMagick/Panel/ rm -rf root/usr/lib/perl5 fi ##we remove all patch from the spec file sed '/Patch/d' $1*.spec > $1.bis.spec sed '/%patch/d' $1.bis.spec > $1.ter.spec mv -f $1.ter.spec $1.spec rm -f $1.bis.spec ###git work if [ ! -e README.md ]; then touch README.md echo "$1 is a contrib for SME Server, a Linux distro oriented server. see http://wiki.contribs.org" > README.md fi ##fill empty directories find * -type d -empty -exec touch {}/.gitignore \; git init git add . git commit -m "first commit to SME Server 9" git co -b sme9 git remote add origin git@github.com:stephdl/$1 git push -u -f origin sme9 ##find the apache authentication echo "" echo "" echo "we are looking if we need to modify the apache authentication" testapache=$(grep -srni 'AuthType Basic' .) if [[ $testapache =~ "AuthType Basic" ]]; then echo "$testapache, go to work" echo 'You must add before $OUT .= " AuthBasicProvider external\n";' else echo "no apache authentication" fi ## help to write the changelog myDate=$(LC_ALL=C date +"%a %b %d %Y") echo "" echo "* $myDate stephane de Labrusse <stephdl@de-labrusse.fr>" echo "- Initial release to sme9" else echo"" echo "###################################" echo "the git directory ~/git_work/$1 is already created." echo "please remove it before to play with the script" echo "think also if you have a github project called stephdl/$1" echo "###################################" fi
build_srpm
a script to build srpm, create the git repository, and export all files to github https://github.com/stephdl/build_srpm . You need to record at /usr/bin/build_srpm (set it executable)
it creates automatically the new repository in github with your credentials (adjust 'stephdl' with your login account and replace 'xxxxxxxxxxxxxx' with your password)
- usage
you must give arguments on the git branch (master, sme9 or anything else) and the package name : build_srpm package_name master give the real name without number of version
build_srpm freedup sme9
###a bit of tests to see if we will not do a mess if [[ -z $1 || -z $2 ]]; then echo "################################################################################################################################" echo "you must give arguments on the git branch (master, sme9 or anything else) and the package name : build_srpm package_name master" echo "################################################################################################################################" exit fi if [[ -d ~/git_work/$1 && -d ~/git_work/$1-sdl ]]; then echo"" echo "###################################" echo "the git directories ~/git_work/$1 and ~/git_work/$1-sdl are already created." echo "please remove it before to play with the script" echo "think also if you have a github project called stephdl/$1 and stephdl/$1-sdl" echo "###################################" exit fi # Create needed dirs on the build box mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,SOURCES,BUILD} mkdir -p ~/exchange rpm -ivh $1*.src.rpm cd ~/rpmbuild/SPECS rpmbuild -bp --nodeps $1*.spec ##we remove all patch from the spec file sed '/Patch/d' $1*.spec > $1.bis.spec sed '/%patch/d' $1.bis.spec > $1.ter.spec mv -f $1.ter.spec $1.spec rm -f $1.bis.spec ###test to see if the directory already exists in ~/git_work, if yes we use ~/git_work/package_name-sdl if [[ ! -d ~/git_work/$1 ]]; then gitexist=0 mkdir -p ~/git_work/$1 NameGitHub=$1 curl -u 'stephdl:xxxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" , "description": "A RPM for SME Server", "homepage": "http://wiki.contribs.org", "private": false, "has_issues": true, "has_wiki": true, "has_downloads": true}' cp -R ~/rpmbuild/BUILD/$1*/* ~/git_work/$1 cp ~/rpmbuild/SPECS/$1.spec ~/git_work/$1 cd ~/git_work/$1 elif [[ -d ~/git_work/$1 ]]; then gitexist=1 mkdir -p ~/git_work/$1-sdl NameGitHub=$1-sdl curl -u 'stephdl:xxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" , "description": "A RPM for SME Server", "homepage": "http://wiki.contribs.org", "private": false, "has_issues": true, "has_wiki": true, "has_downloads": true}' cp -R ~/rpmbuild/BUILD/$1*/* ~/git_work/$1-sdl cp ~/rpmbuild/SPECS/$1.spec ~/git_work/$1-sdl cd ~/git_work/$1-sdl fi ###git work if [ ! -e README.md ]; then touch README.md echo "$1 is a RPM for SME Server, a Linux distro oriented server. see http://wiki.contribs.org" > README.md fi ##fill empty directories find * -type d -empty -exec touch {}/.gitignore \; git init git add . git commit -m "first commit to SME Server" if [[ $2 != 'master' ]]; then git co -b $2 fi if [[ $gitexist = "0" ]]; then git remote add origin git@github.com:stephdl/$1 git push -u -f origin $2 elif [[ $gitexist = "1" ]]; then git remote add origin git@github.com:stephdl/$1-sdl git push -u -f origin $2 fi ##find the apache authentication echo "" echo "" echo "we are looking if we need to modify the apache authentication" testapache=$(grep -srni 'AuthType Basic' .) if [[ $testapache =~ "AuthType Basic" ]]; then echo "$testapache, go to work" echo 'Only for SME9, you must add before $OUT .= " AuthBasicProvider external\n";' else echo "no apache authentication" fi ## help to write the changelog myDate=$(LC_ALL=C date +"%a %b %d %Y") echo "" echo "* $myDate stephane de Labrusse <stephdl@de-labrusse.fr>" echo "- Initial release to SME Server" # a bit of clean rm -rf ~/rpmbuild/SPECS/* ~/rpmbuild/SOURCES/* ~/rpmbuild/SRPMS/* ~/rpmbuild/BUILD/* ## say where we record the source code if [[ $gitexist = "0" ]]; then echo "" echo " packages are in ~/git_work/$1" elif [[ $gitexist = "1" ]]; then echo "" echo " packages are in ~/git_work/$1-sdl" fi
cvs_update
a script to update all CVS entries of smeserver
Launch the script, all smeserver & smecontribs will be updated, adapt the path to your directory and to you cvs access. That script can be run the night to avoid lose time when you must work
usage
cvs_update
#!/bin/bash location=$(pwd) echo "############################" echo " Work In Progress " echo "############################" #first we update the smecontribs tree cd ~/work/smecontribs cvs -z3 -d:ext:smeserver.cvs.sourceforge.net:/cvsroot/smecontribs co -P rpms echo "############################" echo " smecontribs uptodate " echo "############################" echo "############################" echo " Work In Progress " echo "############################" cd ~/work/smeserver cvs -z3 -d:ext:smeserver.cvs.sourceforge.net:/cvsroot/smeserver co -P rpms echo "############################" echo " smeserver uptodate " echo "############################" cd $location
git_srpm
This script is done to import a srpm and to push it after a 'rpmbuild -bp' to your git folder '~/git_work' here.
After that you can commit it if needed
#!/bin/bash ### bit of tests to see if we will not do a mess if [[ -z $1 || -z $2 ]]; then echo "################################################################################################################################" echo "you must give arguments on the git branch (master, sme9, ns6 or anything else) and the package name : git_srpm package_name master" echo "################################################################################################################################" exit fi if [[ -d ~/git_work/$1 ]]; then echo"" echo "###################################" echo "the git directories ~/git_work/$1 is already created." echo "please remove it before to play with the script" echo "think also if you have a github project called stephdl/$1" echo "###################################" exit fi # Create needed dirs on the build box mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,SOURCES,BUILD} mkdir -p ~/exchange mkdir -p ~/git_work/$1 rpm -ivh $1*.src.rpm cd ~/rpmbuild/SPECS rpmbuild -bp --nodeps $1*.spec ##we remove all patch from the spec file # sed '/Patch/d' $1*.spec > $1.bis.spec # sed '/%patch/d' $1.bis.spec > $1.ter.spec # mv -f $1.ter.spec $1.spec # rm -f $1.bis.spec cp -R ~/rpmbuild/BUILD/$1*/* ~/git_work/$1 cp ~/rpmbuild/SPECS/$1.spec ~/git_work/$1 cd ~/git_work/$1 ##fill empty directories find * -type d -empty -exec touch {}/.gitignore \; echo "Now you have to create the correct github remote repository and add the origin" echo " Think to remove all Patches in the specfile" echo "" echo " cd ~/git_work/$1" echo " git init; git add . ; git commit -m 'first commit' ; git checkout -b $2" echo " git remote add origin git@github.com:stephdl/$1.git" echo " git push -u origin $2" rm -f ~/rpmbuild/SPECS/$1*.spec ~/rpmbuild/SOURCES/$1* ~/rpmbuild/SRPMS/$1*.src.rpm
Cron night Job
I don't like to wait so each night I do an init of mock on all configurations I need
* 2 * * * mock -r epel-5-x86_64 --init; mock -r epel-5-i386 --init * 3 * * * mock -r epel-6-x86_64 --init; mock -r epel-6-i386 --init * 4 * * * mock -r smeserver-9-x86_64-base --init; mock -r smeserver-9-i386-base --init * 5 * * * mock -r smeserver-8-x86_64-base --init; mock -r smeserver-8-i386-base --init * 6 * * * /usr/bin/cvs_update
srpm-import to buildsrv
srpm-import is a script to import to buildsrv, you need to be granted to use it. the purpose is to create new source in CVS from a remote srpm
usage:
srpm-import smeserver-contribs contrib9 http://mirror.de-labrusse.fr/smeserver/6/SRPMS/smeserver-php-scl-0.4-3.el6.sme.src.rpm
code:
#!/bin/bash # Author : stephane de labrusse <stephdl@de-labrusse.fr> @2015 # the purpose is to import a srpm from a remote url in the CVS of buildsrv #test if all arguments are here if [[ -z $3 ]]; then echo "########################################################################################################" echo "Import srpm to CVS on buildsrv : srpm-import Folder_Name Branch_Name(smeX|contribsX) Url_2_Remote_Srpm" echo "########################################################################################################" exit fi #we test the branchname if [[ $2 != "sme"[8-9] && $2 != "contribs"[8-9] ]];then echo "" echo "the branchname name is not good, must be sme[8-9] or contribs[8-9]" exit 1 fi #we test the remote url if wget $3 >/dev/null 2>&1 ; then echo "Url : $3 exists..." else echo "" echo "Url : $3 doesn't exists.." exit 1 fi #test root cvs folder if [[ -d $1 ]]; then echo "the source folder $1 already exists" fi #mkdir root cvs folder mkdir -p $1 cvs add $1 cd $1 #test the branch name if [[ -d $2 ]];then echo "" echo "the branch name $1/$2 already exists : we stop the script." echo "" echo "You must import a new tarball manually if needed :" echo " cd $1/$2" echo " wget http://your.src.rpm" echo " rpm2cpio *.src.rpm | cpio -idmv --no-absolute-filenames" echo " make new-sources FILES=*.?z" echo " rm -rf *.src.rpm *.?z *.spec" echo " cvs commit -m 'new source commited for the branch $2 of $1'" exit 1 fi #mkdir branch name (smeX or contribsX) mkdir -p $2 cvs add $2 cd $2 #import correct date of first import echo $(date +%s) > import.log #copy makefile FIXME cp -f ~/Makefile . sed -i -e "s/smeserver-durep/$1/g" Makefile #wget the srpm by remote url wget $3 #untar the srpm rpm2cpio *.src.rpm | cpio -idmv --no-absolute-filenames #import the new source make new-sources FILES=*.?z #a bit of clean rm -rf *.src.rpm *.?z #import all to cvs cvs add * #sed work version=$(grep -sri 'define version' *.spec | sed 's/%define version //gI') release=$(grep -sri 'define release' *.spec | sed 's/%define release //gI') #bump the release release=$((release + 1)) #write to the spec file sed -i "/%define release/c %define release $release" *.spec sed -i "/changelog/a * $(LC_ALL=C date +"%a %b %d %Y") stephane de Labrusse <stephdl@de-labrusse.fr> $version-$release.sme\n- Initial release to $2\n" *.spec echo "" echo "Verify in $1/$2 if all is well, then commit the changes" echo " cvs commit -m '$(LC_ALL=C date +"%a %b %d %Y") stephane de Labrusse <stephdl@de-labrusse.fr> $version-$release.sme - Initial release to $2'" echo "" echo " If it is the first importation of a srpm, you must enable it in the CVSROOT, first you have to refresh the CVS source" echo " cvs co CVSROOT" echo " Then you must edit /CVSROOT/modules and add the name of the new CVS module (alphabetically sorted), after that commit the changes"
Find
Like the name, simply find rpm in buildsrv
#!/bin/bash if -z $1 ; then echo "You must specify a name of rpm" exit fi find {/teams/smeserver,/mirrors/} -name "*$1*"
usage:
./find rpmname