RPM Tools

From SME Server
Revision as of 18:23, 18 January 2015 by Stephdl (talk | contribs)
Jump to navigationJump to search
PythonIcon.png Skill level: Developer
Risk of inconsistencies with Koozali SME Server methodology, upgrades & functionality is high. One must be knowledgeable about how changes impact their Koozali SME Server. Significant risk of irreversible harm.


Rpm Tools

Introduction

In this howto we could let script-tools we need to build rpm

Stephdl

stephdl Stéphane de Labrusse AKA Stephdl

mockbuild

mock get a special syntax not always easy to recall, here a script that you need to record at /usr/bin/mockbuild (set it executable) https://github.com/stephdl/mockbuild

  • mockbuild usage
mockbuild package.src.rpm  mock_configuration

mock_configuration can be found in /etc/mocks 

smeserver-9-x86_64-base smeserver-9-i386-base
smeserver-8-x86_64-base smeserver-8-i386-base

#!/bin/bash
   # who i'm
   ME=$(whoami)
   mkdir -p /home/$ME/exchange

   #test if the second argument is here
   if -z $2 ; then
   echo "################################################"
   echo "you must give as argument the mock configuration"
   echo "   smeserver-9-x86_64-base smeserver-9-i386-base"
   echo "   smeserver-8-x86_64-base smeserver-8-i386-base"
   echo " mockbuild package.src.rpm  mock_configuration  "
   echo "################################################"
   exit
   fi
                                                              
   mock -r $2 --rebuild $1 --resultdir=/home/$ME/exchange/$2
                                          
   # we give the path where are files
   echo ""
   echo "=================================================================================================================="
   echo "RPMS are in /home/$ME/exchange/$2 of your build server"
   echo ""
   ls -1 /home/$ME/exchange/$2/*.rpm
   echo "=================================================================================================================="
                                          
   # and we sign rpm if needed, if not simply do 'enter' with keyboard
   echo ""
   echo "3.Sign your rpms"
   rpm --resign /home/$ME/exchange/$2/*.rpm 2>&1 > /dev/null

A the end I use rpm to sign my rpm, if you don't need it you can remove it.

git_ignore

git don't track empty directory that we may need for a rpm, here a script that you need to record at /usr/bin/git_ignore (set it executable) https://github.com/stephdl/git_ignore

usage in the git directory call the script and track all gew files by " git add . && git commit -m 'your-message' " 

git_ignore

#in order to add a .gitignore file to track empty folder
find * -type d -empty -exec touch {}/.gitignore \;

git_mockbuild

a git integration for mock, git and rpm https://github.com/stephdl/git_mockbuild

you can record this script in /usr/bin 

cp git_mockbuild /usr/bin chmod 775 /usr/bin/git_mockbuild

you need to make a folder with the exact name of the rpm (eg smeserver-dhcpmanager). Inside of this folder you create your git repository and work on files and folder of your RPM. Be aware that GIT do not track empty repositories, you have to create a '.gitignore' (even with nothing inside) in each empty directory, if you want they will be incorporated in your GIT repository.

you can use this command line in case of many folders and subfolders 

find * -type d -empty -exec touch {}/.gitignore \;

Once you want to make the RPM then simply use git_mockbuild without arguments in the root folder of your git repository, the architecture of build come from the spec file. But if needed you can choose another build arch. You have to choose one argument. 

git_mockbuild or git_mockbuild sme8-386 sme8-64 sme9-386 sme9-64

by default if the build architecture is not specified in the spec file, it is the case of packages which are not 'noarch', the RPM will be make by mock with the same architecture of your build server. Therefore you have to specify which arch you want if you need another rpm architecture (eg i386 instead of x86_64)

The CentOS target version comes from the GIT branch you created in your GIT repository. For example if your branch is named sme9 then the build is done for el6, else the target CentOS version is el5.

For an automatic build for all architectures you should take considerations about Plague which is done for that.

Each time you execute the script, the GIT branch in use is tagged with the version of the RPM.

The result of build can be find in the 'exchange' folder of the user's home of that script

  • usage

in the git directory of your rpm (you need to install git & mock installed) 

git_mockbuild

#!/bin/bash
##############################################################################################
## a big thank to daniel berteaud from firewall-service who started in first to write this script
## the purpose of this script is to create a targ.gz from the GIT folder in order to build a rpm
## then after mock is used to make the rpm in a chroot  
##############################################################################################
# git_mockbuild     [ MOCKCONF ]    [ SME_Version ]
# parameters are optional
ME=$(whoami)

# What is the package name
# The package name is given by the name of the main folder
PACKAGE=$(basename $(pwd))

# Informations needed from the spec file
VERSION=$(rpm -q --qf "%{version}\n" --specfile $PACKAGE.spec | head -1)
ARCH=$(rpm -q --qf "%{arch}\n" --specfile $PACKAGE.spec | head -1)
RELEASE=$(rpm -q --qf "%{release}\n" --specfile $PACKAGE.spec | head -1)

# Which git branch we are talking
BRANCH=$(git branch | grep '*' | cut -d' ' -f2)

# Find the centos version
DIST=$2
if -z $DIST && "$BRANCH" =~ "sme9" ; then
   DIST="el6"
elif [ -z $DIST ]; then
   DIST="el5"
fi

# Create a tmp folder in /tmp
TMPDIR=$(mktemp -d)
mkdir -p $TMPDIR/$PACKAGE-$VERSION/

# Tag GIT using the version in the spec file
##we do a hack to retrieve the release without %{?dist}
RELEASETAG=${RELEASE/.el6/}

TAG=$VERSION"-"$RELEASETAG"_"$DIST
git tag -f $TAG 2>&1 > /dev/null

# Create needed dirs on the build box
mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,SOURCES}
mkdir -p ~/exchange


# Generate a tgz archive from the repo
git archive --format=tar --prefix=$PACKAGE-$VERSION/ $BRANCH | tar xf - -C $TMPDIR

# Generate ChangeLog
git log --format=%H | git log --pretty --stdin --no-walk > $TMPDIR/$PACKAGE-$VERSION/CHANGELOG.git

# Create the tar.gz archive 
tar czf $TMPDIR/$PACKAGE-$VERSION.tar.gz -C $TMPDIR $PACKAGE-$VERSION

# cp the spec file and tar.gz
cp $TMPDIR/$PACKAGE-$VERSION.tar.gz ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz
cp $PACKAGE.spec ~/rpmbuild/SPECS/

# And build
echo""
echo "0.Tips"
echo "# git_mockbuild     [ MOCKCONF ]    [ SME_Server_version ]"
echo 'you can choose another build architecture : git_mockbuild sme8-386 sme8-64 sme9-386 sme9-64'
echo 'you can choose which SME_Server target version you want : sme8 sme9'
echo ""
echo "1.Build the src.rpm"
echo ""
cd ~/rpmbuild/SOURCES/  
tar -xvf ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz 2>&1 > /dev/null
rpmbuild -bs --nodeps ~/rpmbuild/SPECS/$PACKAGE.spec
echo ""
echo "2.Build with Mock"
echo ""
echo "=============================================================="
echo "The build RPM architecture is $ARCH" 
echo "The target CentOS version is $DIST"

if [ "$DIST" == "el5" ]; then
   if [ "$ARCH" == "noarch" ]; then
       MOCKCONF="smeserver-8-i386-base"
   elif [ "$ARCH" == "x86_64" ]; then
       MOCKCONF="smeserver-8-x86_64-base"
   elif [ "$ARCH" == "i386" ]; then
       MOCKCONF="smeserver-8-i386-base"
   fi;
elif [ "$DIST" == "el6" ]; then
   if [ "$ARCH" == "noarch" ]; then
       MOCKCONF="smeserver-9-i386-base"
   elif [ "$ARCH" == "x86_64" ]; then
       MOCKCONF="smeserver-9-x86_64-base"
   elif [ "$ARCH" == "i386" ]; then
       MOCKCONF="smeserver-9-i386-base"
   fi;
fi


if ! [ -z "$1" ]; then
   if [ $1 = "sme8-386" ]; then
       MOCKCONF="smeserver-8-i386-base"
   elif [ $1 = "sme8-64" ]; then
       MOCKCONF="smeserver-8-x86_64-base"
   elif [ $1 = "sme9-386" ]; then
       MOCKCONF="smeserver-9-i386-base"
   elif [ $1 = "sme9-64" ]; then
       MOCKCONF="smeserver-9-x86_64-base"
   fi;
fi

echo "The mock chroot configuration used is  $MOCKCONF"
echo "=============================================================="
echo ""
mock -r $MOCKCONF --rebuild ~/rpmbuild/SRPMS/$PACKAGE-$VERSION-*.src.rpm --resultdir=/home/$ME/exchange/$PACKAGE-$VERSION

# a bit of clean

rm -f  ~/rpmbuild/SPECS/$PACKAGE.spec ~/rpmbuild/SOURCES/$PACKAGE-$VERSION.tar.gz ~/rpmbuild/SRPMS/$PACKAGE-$VERSION*.src.rpm
rm -rf $TMPDIR  ~/$PACKAGE-$VERSION 

# we give the path where are files
echo ""
echo "=================================================================================================================="
echo "RPMS are in /home/$ME/exchange/$PACKAGE-$VERSION of your build server"
echo ""
ls -1 ~/exchange/$PACKAGE-$VERSION/*.rpm
echo "=================================================================================================================="

# and we sign rpm if needed, if not simply do 'enter' with keyboard
echo ""
echo "3.Sign your rpms"
rpm --resign ~/exchange/$PACKAGE-$VERSION/*.rpm 2>&1 > /dev/null

contrib_migration

a script to migrate contrib from sme8 CVS to your local git and to your remote Github account for sme9 https://github.com/stephdl/contrib_migration . You need to record at /usr/bin/contrib_migration (set it executable)

simply call the script with the name of the cvs directory. Of course you have to install cvs and retrieve the source code in an anonymous or with a developer access 

contrib_migration smeserver-wordpress

If the directory is not existed in your git directory (~/git_work) in my server, then we launch the script. After that it creates automatically the new repository in github with your credentials (adjust 'stephdl' with your login account and replace 'xxxxxxxxxxxxxx' with your password) After that the script does several things

  • create the local git repository
  • create the remote github account
  • build all patches and remove them
  • move all perl files to the correct new path
  • test if the new apache authentication is needed and where.
  • add a .git_ignore in empty folders (needed by git)
  • track all that stuff :)
  • usage

you need to be outside of the cvs folder (eg in the 'rpms' folder) 

contrib_migration smeserver-packagename

#test if the first argument is here
   if -z $1 ; then
       echo "#################################################################################"
       echo "you must give as argument the folder name in cvs  : contrib_migration folder_name"
       echo "#################################################################################"
       exit
   fi



if  ! -d ~/git_work/$1 ; then

   rm -rf $1/contribs8
   cvs co $1/contribs8
   cd $1/contribs8
   make prep
   NameGitHub=$1
   curl -u 'stephdl:xxxxxxxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" ,  "description": "A contrib for SME9",  "homepage": "http://wiki.contribs.org",  "private": false,  "has_issues": true,  "has_wiki": true,  "has_downloads": true}'
   mkdir -p ~/git_work/$1

   cp -R smeserver-*/* ~/git_work/$1
   cp $1*.spec ~/git_work/$1
   cd ~/git_work/$1

     if [ -e root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/ ]; then 
         mkdir -p root/usr/share/perl5/vendor_perl/esmith/FormMagick/Panel/
         mv root/usr/lib/perl5/site_perl/esmith/FormMagick/Panel/* root/usr/share/perl5/vendor_perl/esmith/FormMagick/Panel/
         rm -rf root/usr/lib/perl5
     fi
##we remove all patch from the spec file
   sed '/Patch/d' $1*.spec > $1.bis.spec
   sed '/%patch/d' $1.bis.spec > $1.ter.spec
   mv -f $1.ter.spec $1.spec
   rm -f $1.bis.spec


###git work 

       if [ ! -e README.md ]; then
       touch README.md
       echo "$1 is a contrib for SME Server, a Linux distro oriented server. see http://wiki.contribs.org" > README.md
       fi

      ##fill empty directories
      find * -type d -empty -exec touch {}/.gitignore \;

   git init
   git add .
   git commit -m "first commit to SME Server 9"
   git co -b sme9
   git remote add origin git@github.com:stephdl/$1
   git push -u -f origin sme9

##find the apache authentication
   echo ""
   echo ""
   echo "we are looking if we need to modify the apache authentication"
   testapache=$(grep -srni 'AuthType Basic' .)
       if $testapache =~ "AuthType Basic" ; then
           echo "$testapache, go to work"
           echo 'You must add before         $OUT .= "    AuthBasicProvider external\n";'
           
       else
           echo "no apache authentication"
       fi

## help to write the changelog
myDate=$(LC_ALL=C date +"%a %b %d %Y")
   echo ""
   echo "* $myDate stephane de Labrusse <stephdl@de-labrusse.fr>"
   echo "- Initial release to sme9"

else
   echo""
   echo "###################################"
   echo "the git directory ~/git_work/$1 is already created."
   echo "please remove it before to play with the script"
   echo "think also if you have a github project called stephdl/$1"
   echo "###################################"
fi

build_srpm

a script to build srpm, create the git repository, and export all files to github https://github.com/stephdl/build_srpm . You need to record at /usr/bin/build_srpm (set it executable)

it creates automatically the new repository in github with your credentials (adjust 'stephdl' with your login account and replace 'xxxxxxxxxxxxxx' with your password)

  • usage

you must give arguments on the git branch (master, sme9 or anything else) and the package name : build_srpm package_name master give the real name without number of version 

###a bit of tests to see if we will not do a mess
   if | -z $2 ; then
       echo "################################################################################################################################"
       echo "you must give arguments on the  git branch (master, sme9 or anything else) and the package name : build_srpm package_name master"
       echo "################################################################################################################################"
       exit
   fi

   if -d ~/git_work/$1 && -d ~/git_work/$1-sdl ; then
       echo""
       echo "###################################"
       echo "the git directories ~/git_work/$1 and  ~/git_work/$1-sdl are already created."
       echo "please remove it before to play with the script"
       echo "think also if you have a github project called stephdl/$1 and stephdl/$1-sdl"
       echo "###################################"
       exit
   fi
# Create needed dirs on the build box

   mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,SOURCES,BUILD}
   mkdir -p ~/exchange

   rpm -ivh $1*.src.rpm
   cd ~/rpmbuild/SPECS
   rpmbuild -bp --nodeps $1*.spec

##we remove all patch from the spec file
   sed '/Patch/d' $1*.spec > $1.bis.spec
   sed '/%patch/d' $1.bis.spec > $1.ter.spec
   mv -f $1.ter.spec $1.spec
   rm -f $1.bis.spec


###test to see if the directory already exists in ~/git_work, if yes we use ~/git_work/package_name-sdl
   if ! -d ~/git_work/$1 ; then

       gitexist=0
       mkdir -p ~/git_work/$1
       NameGitHub=$1
       curl -u 'stephdl:xxxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" , "description": "A RPM for SME Server", "homepage": "http://wiki.contribs.org", "private": false, "has_issues": true, "has_wiki": true, "has_downloads": true}'

       cp -R ~/rpmbuild/BUILD/$1*/* ~/git_work/$1
       cp ~/rpmbuild/SPECS/$1.spec ~/git_work/$1
       cd ~/git_work/$1


   elif -d ~/git_work/$1 ; then

       gitexist=1
       mkdir -p ~/git_work/$1-sdl
       NameGitHub=$1-sdl
       curl -u 'stephdl:xxxxxxxxx' https://api.github.com/user/repos -d '{"name": "'"$NameGitHub"'" , "description": "A RPM for SME Server", "homepage": "http://wiki.contribs.org", "private": false, "has_issues": true, "has_wiki": true, "has_downloads": true}'

       cp -R ~/rpmbuild/BUILD/$1*/* ~/git_work/$1-sdl
       cp ~/rpmbuild/SPECS/$1.spec ~/git_work/$1-sdl
       cd ~/git_work/$1-sdl

   fi


###git work

       if [ ! -e README.md ]; then
       touch README.md
       echo "$1 is a RPM for SME Server, a Linux distro oriented server. see http://wiki.contribs.org" > README.md
       fi

      ##fill empty directories
      find * -type d -empty -exec touch {}/.gitignore \;

   git init
   git add .
   git commit -m "first commit to SME Server"

   if $2 != 'master' ; then
       git co -b $2
   fi
   
   if $gitexist = "0" ; then 
       git remote add origin git@github.com:stephdl/$1
       git push -u -f origin $2

   elif $gitexist = "1" ; then
       git remote add origin git@github.com:stephdl/$1-sdl
       git push -u -f origin $2

   fi


##find the apache authentication
   echo ""
   echo ""
   echo "we are looking if we need to modify the apache authentication"
   testapache=$(grep -srni 'AuthType Basic' .)
       if $testapache =~ "AuthType Basic" ; then
           echo "$testapache, go to work"
           echo 'Only for SME9, you must add before $OUT .= " AuthBasicProvider external\n";'
           
       else
           echo "no apache authentication"
       fi

## help to write the changelog
myDate=$(LC_ALL=C date +"%a %b %d %Y")
   echo ""
   echo "* $myDate stephane de Labrusse <stephdl@de-labrusse.fr>"
   echo "- Initial release to SME Server"

# a bit of clean

   rm -rf ~/rpmbuild/SPECS/* ~/rpmbuild/SOURCES/* ~/rpmbuild/SRPMS/* ~/rpmbuild/BUILD/* 

## say where we record the source code
   if $gitexist = "0" ; then
       echo ""
       echo " packages are in ~/git_work/$1"
   elif $gitexist = "1" ; then
       echo ""
       echo " packages are in ~/git_work/$1-sdl"
   fi
Retrieved from "https://wiki.koozali.org/index.php?title=RPM_Tools&oldid=27270"