Difference between revisions of "Koji Build Farm"

From SME Server
Jump to navigationJump to search
Line 1: Line 1:
 
Official Koji documentation can be found at: https://docs.pagure.org/koji/
 
Official Koji documentation can be found at: https://docs.pagure.org/koji/
  
 
+
{{Note box|This is a work in progress.....
This is a work in progress.....
+
And some components do not work yet}}
 
 
  
 
I'll document what I have done so far, what is working and what is not.
 
I'll document what I have done so far, what is working and what is not.
 
  
 
When the build farm is working, we'll add in how to configure it for building smeserver (packages, repositories and ISOs etc.)
 
When the build farm is working, we'll add in how to configure it for building smeserver (packages, repositories and ISOs etc.)
Line 34: Line 32:
  
 
Disk: 20GB (but I'm only using ~25%)
 
Disk: 20GB (but I'm only using ~25%)
 +
 +
You'll need to set up your network:
 +
 +
Log into your server as root and<syntaxhighlight lang="bash">
 +
nmtui
 +
ip address
 +
ping google.com
 +
</syntaxhighlight>I'd suggest an update is in order<syntaxhighlight lang="bash">
 +
dnf update
 +
 +
</syntaxhighlight>Configure some basic tools and settings<syntaxhighlight lang="bash">
 +
dnf install setools-console
 +
dnf config-manager --set-enabled powertools
 +
dnf install epel-release
 +
dnf install policycoreutils-python-utils
 +
dnf install rsyslog
 +
dnf install cockpit
 +
systemctl enable cockpit.socket --now
 +
systemctl start cockpit.socket
 +
dnf install setroubleshoot-server
 +
setsebool -P allow_httpd_anon_write=1
 +
setsebool -P httpd_can_network_connect_db 1
 +
reboot
 +
</syntaxhighlight>Install koji hub and pre-requisites<syntaxhighlight lang="bash">
 +
dnf install koji-hub mod_ssl
 +
dnf module enable postgresql:10
 +
dnf install postgresql-server
 +
dnf install koji
 +
</syntaxhighlight>We'll be using ssl certificates so let's create the koji ssl working directories and edit the koji ssl config file<syntaxhighlight lang="bash">
 +
mkdir -p /etc/pki/koji/{certs,private,confs}
 +
cd /etc/pki/koji
 +
nano ssl.cnf
 +
 +
</syntaxhighlight>and insert the following into ssl.conf
 +
 +
I suggest you change the defaults in [req_distinguished_name] to yours to make it easier when generating certs....
 +
{{Note box|I suggest you change the defaults in [req_distinguished_name] to yours to make it easier when generating certs....}}<syntaxhighlight lang="ini">
 +
HOME                    = .
 +
RANDFILE                = .rand
 +
 +
[ca]
 +
default_ca              = ca_default
 +
 +
[ca_default]
 +
dir                    = .
 +
certs                  = $dir/certs
 +
crl_dir                = $dir/crl
 +
database                = $dir/index.txt
 +
new_certs_dir          = $dir/newcerts
 +
certificate            = $dir/%s_ca_cert.pem
 +
private_key            = $dir/private/%s_ca_key.pem
 +
serial                  = $dir/serial
 +
crl                    = $dir/crl.pem
 +
x509_extensions        = usr_cert
 +
name_opt                = ca_default
 +
cert_opt                = ca_default
 +
default_days            = 3650
 +
default_crl_days        = 30
 +
default_md              = sha256
 +
preserve                = no
 +
policy                  = policy_match
 +
 +
[policy_match]
 +
countryName            = match
 +
stateOrProvinceName    = match
 +
organizationName        = match
 +
organizationalUnitName  = optional
 +
commonName              = supplied
 +
emailAddress            = optional
 +
 +
[req]
 +
default_bits            = 2048
 +
default_keyfile        = privkey.pem
 +
default_md              = sha256
 +
distinguished_name      = req_distinguished_name
 +
attributes              = req_attributes
 +
x509_extensions        = v3_ca # The extensions to add to the self signed cert
 +
string_mask            = MASK:0x2002
 +
 +
[req_distinguished_name]
 +
countryName                    = Country Name (2 letter code)
 +
countryName_default            = AU
 +
countryName_min                = 2
 +
countryName_max                = 2
 +
stateOrProvinceName            = State or Province Name (full name)
 +
stateOrProvinceName_default    = Victoria
 +
localityName                    = Locality Name (eg, city)
 +
localityName_default            = Melbourne
 +
0.organizationName              = Organization Name (eg, company)
 +
0.organizationName_default      = Koozali
 +
organizationalUnitName          = Organizational Unit Name (eg, section)
 +
commonName                      = Common Name (eg, your name or your server\'s hostname)
 +
commonName_max                  = 64
 +
emailAddress                    = Email Address
 +
emailAddress_max                = 64
 +
 +
[req_attributes]
 +
challengePassword              = A challenge password
 +
challengePassword_min          = 4
 +
challengePassword_max          = 20
 +
unstructuredName                = An optional company name
 +
 +
[usr_cert]
 +
basicConstraints                = CA:FALSE
 +
nsComment                      = "OpenSSL Generated Certificate"
 +
subjectKeyIdentifier            = hash
 +
authorityKeyIdentifier          = keyid,issuer:always
 +
 +
[v3_ca]
 +
subjectKeyIdentifier            = hash
 +
authorityKeyIdentifier          = keyid:always,issuer:always
 +
basicConstraints                = CA:true
 +
</syntaxhighlight>Create the ca key for the server<syntaxhighlight lang="bash">
 +
touch index.txt
 +
echo 01 > serial
 +
openssl genrsa -out private/koji_ca_cert.key 2048
 +
openssl req -config ssl.cnf -new -x509 -days 3650 -key private/koji_ca_cert.key -out koji_ca_cert.crt -extensions v3_ca
 +
</syntaxhighlight>Create a script to make certs<syntaxhighlight lang="bash">
 +
mkdir -p ~/bin
 +
nano ~/bin/koji_make_cert.sh
 +
</syntaxhighlight>and add the following<syntaxhighlight lang="bash">
 +
#!/bin/bash
 +
# if you change your certificate authority name to something else you will
 +
# need to change the caname value to reflect the change.
 +
caname=koji
 +
 +
# user is equal to parameter one or the first argument when you actually
 +
# run the script
 +
user=$1
 +
 +
openssl genrsa -out private/${user}.key 2048
 +
cat ssl.cnf | sed 's/insert_hostname/'${user}'/'> ssl2.cnf
 +
openssl req -config ssl2.cnf -new -nodes -out certs/${user}.csr -key private/${user}.key
 +
openssl ca -config ssl2.cnf -keyfile private/${caname}_ca_cert.key -cert ${caname}_ca_cert.crt \
 +
    -out certs/${user}.crt -outdir certs -infiles certs/${user}.csr
 +
cat certs/${user}.crt private/${user}.key > ${user}.pem
 +
mv ssl2.cnf confs/${user}-ssl.cnf
 +
</syntaxhighlight>and make it executable<syntaxhighlight lang="bash">
 +
chmod a+x ~/bin/koji_make_cert.sh
 +
</syntaxhighlight>Lets create some certificates and add our admin user<syntaxhighlight lang="bash">
 +
koji_make_cert.sh kojihub
 +
koji_make_cert.sh kojiweb
 +
koji_make_cert.sh kojira
 +
koji_make_cert.sh kojid
 +
koji_make_cert.sh kojiadmin
 +
useradd kojiadmin
 +
</syntaxhighlight>We need to be the kojiadmin user to get the right permissions when we copy over the required certs, so...<syntaxhighlight lang="bash">
 +
su - kojiadmin
 +
mkdir ~/.koji
 +
cp /etc/pki/koji/kojiadmin.pem ~/.koji/client.crt  # NOTE: It is IMPORTANT you use the PEM and NOT the CRT
 +
cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/clientca.crt
 +
cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/serverca.crt
 +
exit
 +
</syntaxhighlight>

Revision as of 07:04, 18 July 2023

Official Koji documentation can be found at: https://docs.pagure.org/koji/


Important.png Note:
This is a work in progress.....

And some components do not work yet


I'll document what I have done so far, what is working and what is not.

When the build farm is working, we'll add in how to configure it for building smeserver (packages, repositories and ISOs etc.)

Building blocks

A Koji Build farm is comprised of a number of components that work together.

Major Koji components:

  • hub
  • web server
  • build servers
  • build daemon
  • Dnf|Yum repository creation and maintenance daemon

In our build, we will have only 2 servers.

  • hub - which will run the hub, web, build daemon and def|Yum repository daemon
  • build server - there can be multiple of these, but we'll just do 1 to start with

These servers will be based on bare Rocky 8 - minimal install, servers.

Hub/Web Server

OS: Rocky 8.8-minimal

Memory: 8GB

Disk: 20GB (but I'm only using ~25%)

You'll need to set up your network:

Log into your server as root and

nmtui
ip address
ping google.com

I'd suggest an update is in order

dnf update

Configure some basic tools and settings

dnf install setools-console
dnf config-manager --set-enabled powertools
dnf install epel-release
dnf install policycoreutils-python-utils
dnf install rsyslog
dnf install cockpit
systemctl enable cockpit.socket --now
systemctl start cockpit.socket
dnf install setroubleshoot-server
setsebool -P allow_httpd_anon_write=1
setsebool -P httpd_can_network_connect_db 1
reboot

Install koji hub and pre-requisites

dnf install koji-hub mod_ssl
dnf module enable postgresql:10
dnf install postgresql-server
dnf install koji

We'll be using ssl certificates so let's create the koji ssl working directories and edit the koji ssl config file

mkdir -p /etc/pki/koji/{certs,private,confs}
cd /etc/pki/koji
nano ssl.cnf

and insert the following into ssl.conf

I suggest you change the defaults in [req_distinguished_name] to yours to make it easier when generating certs....

Important.png Note:
I suggest you change the defaults in [req_distinguished_name] to yours to make it easier when generating certs....


HOME                    = .
RANDFILE                = .rand

[ca]
default_ca              = ca_default

[ca_default]
dir                     = .
certs                   = $dir/certs
crl_dir                 = $dir/crl
database                = $dir/index.txt
new_certs_dir           = $dir/newcerts
certificate             = $dir/%s_ca_cert.pem
private_key             = $dir/private/%s_ca_key.pem
serial                  = $dir/serial
crl                     = $dir/crl.pem
x509_extensions         = usr_cert
name_opt                = ca_default
cert_opt                = ca_default
default_days            = 3650
default_crl_days        = 30
default_md              = sha256
preserve                = no
policy                  = policy_match

[policy_match]
countryName             = match
stateOrProvinceName     = match
organizationName        = match
organizationalUnitName  = optional
commonName              = supplied
emailAddress            = optional

[req]
default_bits            = 2048
default_keyfile         = privkey.pem
default_md              = sha256
distinguished_name      = req_distinguished_name
attributes              = req_attributes
x509_extensions         = v3_ca # The extensions to add to the self signed cert
string_mask             = MASK:0x2002

[req_distinguished_name]
countryName                     = Country Name (2 letter code)
countryName_default             = AU
countryName_min                 = 2
countryName_max                 = 2
stateOrProvinceName             = State or Province Name (full name)
stateOrProvinceName_default     = Victoria
localityName                    = Locality Name (eg, city)
localityName_default            = Melbourne
0.organizationName              = Organization Name (eg, company)
0.organizationName_default      = Koozali
organizationalUnitName          = Organizational Unit Name (eg, section)
commonName                      = Common Name (eg, your name or your server\'s hostname)
commonName_max                  = 64
emailAddress                    = Email Address
emailAddress_max                = 64

[req_attributes]
challengePassword               = A challenge password
challengePassword_min           = 4
challengePassword_max           = 20
unstructuredName                = An optional company name

[usr_cert]
basicConstraints                = CA:FALSE
nsComment                       = "OpenSSL Generated Certificate"
subjectKeyIdentifier            = hash
authorityKeyIdentifier          = keyid,issuer:always

[v3_ca]
subjectKeyIdentifier            = hash
authorityKeyIdentifier          = keyid:always,issuer:always
basicConstraints                = CA:true

Create the ca key for the server

touch index.txt
echo 01 > serial
openssl genrsa -out private/koji_ca_cert.key 2048
openssl req -config ssl.cnf -new -x509 -days 3650 -key private/koji_ca_cert.key -out koji_ca_cert.crt -extensions v3_ca

Create a script to make certs

mkdir -p ~/bin
nano ~/bin/koji_make_cert.sh

and add the following

#!/bin/bash
# if you change your certificate authority name to something else you will
# need to change the caname value to reflect the change.
caname=koji

# user is equal to parameter one or the first argument when you actually
# run the script
user=$1

openssl genrsa -out private/${user}.key 2048
cat ssl.cnf | sed 's/insert_hostname/'${user}'/'> ssl2.cnf
openssl req -config ssl2.cnf -new -nodes -out certs/${user}.csr -key private/${user}.key
openssl ca -config ssl2.cnf -keyfile private/${caname}_ca_cert.key -cert ${caname}_ca_cert.crt \
    -out certs/${user}.crt -outdir certs -infiles certs/${user}.csr
cat certs/${user}.crt private/${user}.key > ${user}.pem
mv ssl2.cnf confs/${user}-ssl.cnf

and make it executable

chmod a+x ~/bin/koji_make_cert.sh

Lets create some certificates and add our admin user

koji_make_cert.sh kojihub
koji_make_cert.sh kojiweb
koji_make_cert.sh kojira
koji_make_cert.sh kojid
koji_make_cert.sh kojiadmin
useradd kojiadmin

We need to be the kojiadmin user to get the right permissions when we copy over the required certs, so...

su - kojiadmin
mkdir ~/.koji
cp /etc/pki/koji/kojiadmin.pem ~/.koji/client.crt   # NOTE: It is IMPORTANT you use the PEM and NOT the CRT
cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/clientca.crt
cp /etc/pki/koji/koji_ca_cert.crt ~/.koji/serverca.crt
exit