Howto interact with gitea and koji to do an rpm build for SME11

Introduction

Unlike SME10 and earlier, SME11 is being developed using cutting edge 21st century tools. This means that the (open) source code and associated file are stored in GIT repositories, surfaced through a Gitea server which provides a web based User Interface as well as an API. Rpm and ISO builds are done using the Koji tool chain which has been developed for the Fedora distribution, but has become well used for other Open Source projects.

The intention is to mirror the Gitea sources to Github so that anyone can examine and propose changes. This is still a work in progress https://github.com/smeserver . You can skip the stuff about CVS unless you want access the old SME10 code.

Setting up your build system

SME11 is based on Rocky 8, so the build system is best based on rocky 8 as well. It could be an SME11 system, but as we are still putting that together, the best choice for now is a minimal Rocky 8 system. This can be installed in a VM under ProxMox, VirtualBox or similar.

Suggest at least 4gb RAM and 50-100gb of Hard Disk.

Configure it according to your own preferences (lots of advice available on the web). Some more specific advice for SME11 building is here .

Here is the relevant directory structure:

├── GITFiles
│   ├── smecontribs
│   │   ├── common
│   │   └── smeserver-mailstats
│   ├── smeserver
│   │   ├── anaconda
│   │   ├── bglibs
│   │   ├── buffer
│   │   ├── buildsys-macros
│   │   ├── checkpassword-pam
│   │   ├── clamav
│   │   ├── common
....
│   │   ├── smeserver-lib-compspec
│   │   ├── smeserver-locale
│   │   ├── smeserver-LPRng
│   │   ├── smeserver-manager
│   │   ├── smeserver-manager-jsquery
│   │   ├── smeserver-manager-locale
│   │   ├── smeserver-mock
│   │   ├── smeserver-mysql
│   │   ├── smeserver-ntp
│   │   ├── smeserver-nutUPS
│   │   ├── smeserver-oidentd
│   │   ├── smeserver-openssh
│   │   ├── smeserver-packetfilter
│   │   ├── smeserver-php
│   │   ├── smeserver-pop3
....
│   │   ├── xlhtml
│   │   └── zsync

"GITfiles" can be anything you want "rpms" was the previous standard.

Note the need for the common directory under each of the organisations

Gitea

The Gitea server is available here.If you are serious about developing systems for SMEServer, then you will need a user. There is a central authorization system, and you can be entered into it by a request to contact@koozali.org

There are two relevant organisations on the gitea server smeserver and smecontribs. One for the core components and one for the contribs. These are just a useful way or categorizing and grouping repositories.

The common directories are necessary if you are to do either local rpm builds or insert build requests into koji. The common directory contains the necessary makefile code.

Install them:

#!/bin/bash

# Define an array of repository organisations you want to clone
repos=("smeserver" "smecontribs")

# Loop through each repository
for repo in "${repos[@]}"; do
  # Navigate to the GITFiles directory
  cd ~/GITFiles/$repo || { echo "Directory ~/GITFiles/$repo does not exist!"; exit 1; }
  
  # Clone the repository
  git clone https://src.koozali.org/$repo/common.git
done

To bring down a specific package, the commands are:

cd smeserver #or smecontribs note this is the directory above the package directory
git clone https://src.koozali.org/<organisation>/<packagename>.git

This will create a directory underneath which will then contain the complete package source.

Note that git clone will not overwrite a previous directory containing an earlier (or the same) version of the package. This is so that you do not inadvertently overwrite precious changes you might have made.

To check that all changes have been written to git, run git status in the directory and it will show you (in red) all files which are different from the ones on the gitea server.

Koji

The Koozali Koji server is here. Like the gitea server, to do anything useful you need to be able to login in to it. The same username/password will work as for the gitea server. Koji is a fully featured build system which we will use for creating rpms and also generating ISO for releases. However the latter has not yet been completed.

Information is here about setting up your build system to allow the use of Koji.

Editing the Source Code

As described above you can pull down the repo for a specific package using the git clone command. Or you can use the gitea web interface to grab which ever file you want and edit it as required, even using the provided gitea editor. You may be able to configure your desktop editor to interact with the remote gitea/git. You can then commit the changes using your editor or the gitea editor/web pages.

My preferred modus operandi is to use a running (and fully updated) SME11 VM to which I have full file access from my desktop using the sshfs package. This allows me to make any changes using my preferred source code editor (geany in my case) and fully test any changes in place, and then copy the changed files back to the build system ready for local rpm build and test before updating to gitea/git and then doing a koji build. YMMV.

Local rpm build

A local rpm build can be achieved using mock. You can do it by issuing the following command:

make mockbuild

While positioned in the cloned directory. mock will read the data in the directory (NOT from git) and create a src and binary rpm. This can then be copied to the test VM and tried.

Saving the Source code to Gitea

All these commands take place while positioned in the cloned directory for the package.

git status is very useful to see what changes have been made as compared to the version in the remote gitea/git server. The changed files are flagged in red before they have been locally staged and green afterwards (staged = call git add).

git pull can be used to bring any existing clone directory up to the latest on the server. However if the local version has been changed since the original clone and not pushed back and therefore deviates from the remote version then an error message is seen and the pull will fail. Unless you have good reason to retain your local changes, the easiest way is to delete the local directory and re-clone.

git all --all will add all the local changes to the list to be pushed back to the repo. You can also use git add to just add one file. Also note that the contents of .gitignore can be edited to cause file types or names to be ignored by the --all parameter.

git commit -m "<comment>" This will prepare to push the changed files back to the remote repo.

git push WILL ACTUALLY do the update to the remote repo.

Once I have saved a set of changes to gitea/git I then paste a link to the set of changes (taken from the Rocket Gitea channel typically) into the corresponding bugzilla entry for the change.

Tagging and starting a build

When you want to do a koji build, then you need to generate a release tag in the gitea/git repo and then submit the current source to koji for the build. This is best done after you have saved any changes etc by a sequence of git add, git commit and git push.

The command to tag and build is:

make tag build

Once the build starts, the command returns a link to the build log, this can then be posted to the corresponding bugzilla record.

Monitoring your build

Lastly you can then watch your build by refreshing the koji web page, or looking for the build on the Rocket Chat channel (once we get that working!).