User talk:Mmccarn

Install mariadb 'alongside' mysql

MariaDB is a drop in place replacement for MySQL, but you can also install it alongside MySQL.

Installing another SQL database server 'alongside' the built-in SME MySQL server allows a SME admin to use web apps that are incompatible with the Centos/SME version of MySQL without making potentially destabilizing changes to core server components.

MariaDB or MySQL can both be installed in an alongside or side-by-side configuration, but MariaDB maintains binary executables compatible with CentOS 5 & 6 and MySQL does not (at least, I could not find any).

Installation

These installation notes are taken from https://mariadb.com/kb/en/installing-mariadb-alongside-mysql/

IMPORTANT: Do not use yum for an 'alongside' ('side-by-side') installation of mariadb.

Create service account

Create a user named mariadb in server-manager. This user account will be used by mariadb when running, and does not need to have a password set.

SME will create a group of the same name automatically.

Download MariaDB and extract into /opt

mkdir -p ~/addons
cd ~/addons
RELEASE=5.5.34
wget https://downloads.mariadb.org/interstitial/mariadb-$RELEASE/kvm-bintar-hardy-x86/mariadb-$RELEASE-linux-i686.tar.gz/from/http://mirror.jmu.edu/pub/mariadb
cd /opt
tar zxvf ~/addons/mariadb-$RELEASE-linux-i686.tar.gz

Create folders and links

RELEASE=5.5.34
cd /opt
ln -s mariadb-$RELEASE-linux-i686 mariadb
mkdir mariadb-data
#
mkdir -p /var/run/mariadb

Setup my.cnf and init.d

Important: On a SME server, 'pid-file' must be specified in /opt/mariadb-data/my.cnf to override the SME default value in /etc/my.cnf

/opt/mariadb-data/my.cnf

'cp' -f /opt/mariadb/support-files/my-medium.cnf /opt/mariadb-data/my.cnf
sed -i -e '/^port.*3306$/ s/= 3306/= 3307/' /opt/mariadb-data/my.cnf
sed -i -e '/^socket.*= \/tmp\/mysql.sock/ s~= /tmp/mysql.sock~= /opt/mariadb-data/mariadb.sock~' /opt/mariadb-data/my.cnf
#
BASEDIR='basedir\t\t= /opt/mariadb'
DATADIR='datadir\t\t= /opt/mariadb-data'
USER='user\t\t= mariadb'
PIDFILE='pid-file\t= /var/run/mariadb/mariadb.pid'
sed -i -e "/^\[mysqld\]$/ s~\[mysqld\]~\[mysqld\]\n$DATADIR\n$BASEDIR\n$USER\n$PIDFILE~" /opt/mariadb-data/my.cnf

Note: The last 'sed' command will add lines for basedir, datadir, user, and pid into my.cnf every time it is run, even if these lines already exist.

/etc/rc.d/init.d/mariadb

'cp' -f /opt/mariadb/support-files/mysql.server /etc/rc.d/init.d/mariadb
sed -i "s~^# Provides: mysql$~# Provides: mariadb~" /etc/rc.d/init.d/mariadb
sed -i "s~^basedir=$~basedir=/opt/mariadb~"          /etc/rc.d/init.d/mariadb
sed -i "s~^datadir=$~datadir=/opt/mariadb-data~"      /etc/rc.d/init.d/mariadb
sed -i "s~lockdir/mysql~lockdir/mariadb~"         /etc/rc.d/init.d/mariadb
sed -i "s~bindir/mysqld_safe\ --datadir~bindir/mysqld_safe\ --defaults-file=/opt/mariadb-data/my.cnf\ --datadir~" /etc/rc.d/init.d/mariadb

file and folder ownership

RELEASE=5.5.34
chown -R mariadb:mariadb mariadb-data mariadb mariadb-$RELEASE-linux-i686
chown mariadb:mariadb /var/run/mariadb

Initialize Environment

cd /opt/mariadb
scripts/mysql_install_db --defaults-file=/opt/mariadb-data/my.cnf

Start Server

/etc/rc.d/init.d/mariadb start

set mariadb root password

A freshly installed mariadb server has no password set for the root user.

In order secure your mariadb server yet easily manage mariadb set the root password for mariadb to match the SME mysql root password.

SME has a template-driven sql command specifically for setting the root password:

CONNECT=--socket=/opt/mariadb-data/mariadb.sock
mysql $CONNECT -password="" < /var/service/mysqld/set.password

Testing

If you set the mariadb root password as shown above you can verify the versions of mysql and mariadb as described in the howto:

# show mysql version
mysql -e "SELECT VERSION();"

# show mariadb version
CONNECT=--socket=/opt/mariadb-data/mariadb.sock
mysql -e "SELECT VERSION();" $CONNECT

Here are alternative ways to show the mariadb version:

# show mariadb version using TCP on port 3307
CONNECT="--protocol=TCP --port=3307"
mysql -e "SELECT VERSION();" $CONNECT

# command lines to check mariadb versions
mysql -e "SELECT VERSION();" --protocol=TCP --port=3307
mysql -e "SELECT VERSION();" --socket=/opt/mariadb-data/mariadb.sock

Configure to start at boot

cd /etc/init.d
chkconfig --add mariadb 
chkconfig --levels 3 mariadb on
# SME Server uses runlevel 7...
cp /etc/rc3.d/S64mariadb /etc/rc7.d

Raw notes

#####################################################################################
# BEGIN
#####################################################################################
# yum repos aren't useful; use the side-by-side install
# Install mariadb side-by-side with mysql on SME Server v8
# From: https://mariadb.com/kb/en/installing-mariadb-alongside-mysql/
#
# create user 'mariadb' in server-manager
#
cd ~
mkdir -p addons
cd addons
RELEASE=5.5.34
wget https://downloads.mariadb.org/interstitial/mariadb-$RELEASE/kvm-bintar-hardy-x86/mariadb-$RELEASE-linux-i686.tar.gz/from/http://mirror.jmu.edu/pub/mariadb
cd /opt
tar zxvf ~/addons/mariadb-$RELEASE-linux-i686.tar.gz
ln -s mariadb-$RELEASE-linux-i686 mariadb
mkdir mariadb-data
cp mariadb/support-files/my-medium.cnf mariadb-data/my.cnf
#
# edit my.cnf as described on the mariadb-alongside-mysql howto (link above)
chown -R mariadb:mariadb mariadb-data mariadb mariadb-$RELEASE-linux-i686
#
cp mariadb/support-files/mysql.server /etc/init.d/mariadb
chmod +x /etc/init.d/mariadb
#
# edit /etc/init.d/mariadb as described in the howto
#
cd /opt/mariadb
scripts/mysql_install_db --defaults-file=/opt/mariadb-data/my.cnf
cd /etc/init.d
chkconfig --add mariadb 
chkconfig --levels 3 mariadb on
# SME Server uses runlevel 7...
mv /etc/rc3.d/S64mariadb /etc/rc7.d
#
# customizations not mentioned in the howto:
mkdir /var/run/mariadb
chown mariadb:mariadb /var/run/mariadb
#
# customize the execution line in /etc/init.d/mariadb
#
# version from howto:
#  $bindir/mysqld_safe --defaults-file=/opt/mariadb-data/my.cnf --datadir="$datadir" --pid-file="$mysqld_pid_file_path" $other_args >/dev/null 2>&1 &
#
# version I'm using:
# $bindir/mysqld_safe --defaults-file=/opt/mariadb-data/my.cnf --datadir="/opt/mariadb-data" --pid-file="/var/run/mariadb/mariadb.pid" $other_args >/dev/null 2>&1 &
#
# Finally, testing access requires asking for a password, then entering an empty password (SME feeds the 'root'@'localhost' password to mysql by default:
# mysql -e "SELECT VERSION();" --port=3307 --protocol=TCP -p
#
# set root password in mariadb same as root password in mysql
mysql --port=3307 --protocol=TCP -p < /var/service/mysqld/set.password
#
# Now the example version check from the howto works:    
mysql -e "SELECT VERSION();" --socket=/opt/mariadb-data/mariadb.sock

Gotchas

my.cnf

I have not copied the customization instructions for my.cnf from the mariadb howto page.

Be sure to make all of the changes to my.cnf suggested.

https://mariadb.com/kb/en/installing-mariadb-alongside-mysql/

/var/run/mariadb

I could not get mariadb to run until I manually created the folder for the process id file.

mkdir /var/run/mariadb
chown mariadb:mariadb /var/run/mariadb

/etc/rc.d/init.d/mariadb

After adding pid-file = /var/run/mariadb/mariadb.pid in /opt/mariadb-data/my.cnf the howto command line worked.

~~I was unable to make the 'mysqld_safe' command line from the howto work. Where the howto wanted this command:~~

~~$bindir/mysqld_safe --defaults-file=/opt/mariadb-data/my.cnf --datadir="$datadir" --pid-file="$mysqld_pid_file_path" $other_args >/dev/null 2>&1 &~~

I needed to specify the datadir instead of using the "$datadir" variable:

~~$bindir/mysqld_safe --defaults-file=/opt/mariadb-data/my.cnf --datadir="/opt/mariadb-data" --pid-file="/var/run/mariadb/mariadb.pid" $other_args >/dev/null 2>&1 &~~

~~If I use the line suggested in the HOWTO, mariadb never starts (although it reports starting on the command line, it records an error in /opt/mariadb-data/<hostname>.err)~~

Backup & Restore

(RequestedDeletion) Nice notes. Would it be worth it to look at the pre-backup and pre-restore events (And config db settings) so MariaDB databases are being backup up ?

/etc/e-smith/events/actions/mariadb-dump-tables

Based on /etc/e-smith/events/actions/mysql-dump-tables
"CONNECT" options separated out for easy modification and potential movement into db variables
"-x" option added to the mysqldump command line to avoid an error about locking log files

#!/bin/sh
#CONNECT=--socket=/opt/mariadb-data/mariadb.sock
CONNECT="--protocol=TCP --port=3307"

if ! $(mysqladmin $CONNECT ping >/dev/null 2>&1)
then
    echo "mariadb is not running - no tables dumped" >&2
    exit 0
fi

mkdir -p /home/e-smith/db/mariadb
for db in $(mysql $CONNECT -BNre "show databases;")
do
    mysqldump $CONNECT  -x --add-drop-table  -QB "$db" -r /home/e-smith/db/mariadb/"$db".dump  || exit 1
done

/etc/e-smith/events/actions/mariadb-load-tables

based on /etc/e-smith/events/actions/mysql-load-tables
doesn't actually load the dbs
NEEDS EXAMINATION!

#!/bin/sh
#CONNECT=--socket=/opt/mariadb-data/mariadb.sock
CONNECT="--protocol=TCP --port=3307"
MARIADATA=/opt/mariadb-data

if ! $(mysqladmin $CONNECT ping >/dev/null 2>&1)
then    
    echo "mariadb is not running - no tables restored" >&2
    exit 0
fi

if [ ! -f $MARIADATA/mysql/user.frm ]
then
    mkdir -p /etc/e-smith/mariadb/init
    for db in $(ls /home/e-smith/db/mariadb/*.dump 2> /dev/null | grep -v '/mysql.dump')
    do
        mv $db /etc/e-smith/mariadb/init/01_$(basename $db .dump).sql
    done
fi

Automation

Backup

Link mariadb-dump-tables into the pre-backup event
Dumped tables should be included in backups as they are stored under /home/e-smith

cd /etc/e-smith/events/pre-backup
ln -s ../actions/mariadb-dump-tables S20mariadb-dump-tables

Restore - NEEDS WORK

The SME Server mysql restore is complicated by various factors that may not apply to a mariadb "alongside" installation. Here's what would need to be done to restore all mariadb databases:

Reinstall mariadb, including setting the password to match the mysql root password
Restore the 'dump' files created during pre-backup individually using:

cd /home/e-smith/db/mariadb
CONNECT=--socket=/opt/mariadb-data/mariadb.sock
mysql $CONNECT < <dbname>.dump

Install Moodle 2.6 using git

Requirements

Recommended minimum browser: recent Google Chrome, recent Mozilla Firefox, Safari 6, Internet Explorer 9 (IE 10 required for drag and drop of files from outside the browser into Moodle)
Moodle upgrade: Moodle 2.2 or later (if upgrading from earlier versions, you must upgrade to 2.2.11 as a first step)
Minimum DB versions: PostgreSQL 8.3, MySQL 5.1.33, MariaDB 5.3.5, MSSQL 2005 or Oracle 10.2
Minimum PHP version: PHP 5.3.3 (always use latest PHP 5.4.x or 5.5.x on Windows - http://windows.php.net/download/)
New recommended PHP extensions: zlib, OPcache

DB Version

SME Server 8.x comes with MySQL v5.0.95. In order to install Moodle without risking destabilizing a SME server by changing the MySQL version, you can install MariaDB 5.3.54 alongside MySql.

OPcache

Zend OPcache is built-in to PHP 5.5, and can be compiled to work with PHP 5.3.3.

I have not been able to find a source online for an RPM for OPcache.

I believe this will make Moodle run more slowly than it would *with* OPcache.

Installation

Prepare your server

Install useful php modules

During installation, Moodle will request php-soap, php-xmlrpc and php-intl. These are all available from the 'smeaddons' repository, and can be installed using:

yum install php-soap php-xmlrpc php-intl

Create an ibay

Create an ibay named 'moodle' in server-manager
Customize some of the settings on the new moodle ibay

IBAY=moodle
/sbin/e-smith/db accounts setprop $IBAY \
FollowSymLinks enabled \
CgiBin enabled \
AllowOverride All \
Group www \
PublicAccess global \
PHPBaseDir "/home/e-smith/files/ibays/$IBAY/:/tmp/" \
UserAccess wr-group-rd-everyone
/sbin/e-smith/signal-event remoteaccess-update
#
mkdir /home/e-smith/files/ibays/$IBAY/moodledata
chown www:www /home/e-smith/files/ibays/$IBAY/moodledata
#

Create a database

Install Mariadb alongside mysql
create a mariadb database for moodle

# Generate a random 23 character password
DBPASS=`< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c23`
DBNAME=moodle
DBUSER=moodle
CONNECT=--socket=/opt/mariadb-data/mariadb.sock
#
echo ;\
echo ;\
echo Creating Database using: ;\
echo DBNAME=$DBNAME ;\
echo DBUSER=$DBUSER ;\
echo DBPASS=$DBPASS ;\
echo ;\
echo Save this information!  You will need it later during initial application setup! \(press \<enter\> when ready\) ;\
read
#
mysql $CONNECT -e "create database $DBNAME; 
grant all privileges on $DBNAME.* to $DBUSER@localhost identified by \"$DBPASS\" with grant option;
quit"

Install Moodle

Download & Checkout using git

    #
    # Download moodle using git
    # http://docs.moodle.org/26/en/Git_for_Administrators#Obtaining_the_code_from_Git
    #
IBAY=moodle
cd /home/e-smith/files/ibays/$IBAY
mv html html.`date +%F-%H%M%S`
git clone git://git.moodle.org/moodle.git html
cd html
git branch -a
git branch --track MOODLE_26_STABLE origin/MOODLE_26_STABLE
git checkout MOODLE_26_STABLE
#
# correct ownership
signal-event ibay-modify $IBAY

Run the moodle installer

#
# run the moodle command line installer
# if prompted, set:
    # mysql port: 3307
    # mysql socket: /opt/mariadb-data/mariadb.sock
    #
cd /home/e-smith/files/ibays/$IBAY/html/admin/cli
sudo -u www /usr/bin/php install.php

Correct database settings if necessary

If you were not prompted for database socket, port, or other connection settings during the command line setup, you will need to correct the settings manually.

The database connection settings are stored in this file:

/home/e-smith/files/ibays/moodle/html/config.php

If you have just run the above database connection commands in the same putty session, you can correct your moodle settings using:

IBAY=moodle
sed -i  s/dbname.*/dbname\ \ \ \ \=\ \'$DBNAME\'\;/  /home/e-smith/files/ibays/$IBAY/html/config.php
sed -i  s/dbuser.*/dbuser\ \ \ \ \=\ \'$DBUSER\'\;/  /home/e-smith/files/ibays/$IBAY/html/config.php
sed -i  s/dbpass.*/dbpass\ \ \ \ \=\ \'$DBPASS\'\;/  /home/e-smith/files/ibays/$IBAY/html/config.php

Optional Settings

Scan Uploads using ClamAV

Moodle can be configured to scan all user files when uploaded.

mysql $CONNECT  -e "use moodle;
update mdl_config set value=1 where name='runclamonupload';
update mdl_config set value='/usr/bin/clamscan' where name='pathtoclam';
quit"

Authentication Settings

To configure authentication mechanisms:

Login to Moodle using an account with administrative rights
Select 'Site Administration'
- Select 'Plugins'
  - Select 'Authentication'

Moodle can be configured to authenticate users using any of the methods listed below:

Manual accounts
No login
CAS server (SSO)
Email-based self-registration
External database
FirstClass server
IMAP server
- Select 'imapcert' if your IMAP server uses a self-signed certificate
LDAP server
MNet authentication
NNTP server
No authentication
PAM (Pluggable Authentication Modules)
POP3 server
RADIUS server
Shibboleth
Web services authentication

Create Additional Administrators

http://docs.moodle.org/26/en/Assign_admins

TiddlyWiki5 Using Node.js

TiddlyWiki is "a complete interactive wiki in JavaScript."

	Warning:
	The install and update routines shown here are based on notes from a working installation. The removal routines are untested.

Prerequisites

node.js > 8.x (note: I was unable to find a binary installer for curent node releases; I use 'gcc' and compile locally)
npm

Assumptions

wiki content will be stored in /opt/tiddlywiki/tiddlers
tiddlywiki code will be stored in /opt/tiddlywiki/node_modules
tiddlywiki will run as user 'www'
tiddlywiki logs will be run as 'smelog'
tiddlywiki will be daemonized using daemontools

Installation

mkdir /opt/tiddlywiki
cd /opt/tiddlywiki
npm install tiddlywiki
chown -R www:www /opt/tiddlywiki/.

Create daemontools scripts, folders, etc

The code below is designed to be run by copy/paste into a server console prompt.

mkdir -p /var/service/tiddlywiki/log
mkdir -p /var/log/tiddlywiki
chown -R smelog:smelog /var/log/tiddlywiki
cd /service
ln -s /var/service/tiddlywiki .
cd /var/service/tiddlywiki
touch down
    #
    # create the service 'run' file
    #
echo '#!/bin/sh
#
# setup node environment
#
exec 2>&1
#
APP_DIR=/opt/tiddlywiki
USER=www
#
NODE_EXEC=/usr/local/bin/node
NODE_ENV=production
NODE_CONFIG_DIR=$APP_DIR
NODE_APP=node_modules/tiddlywiki/tiddlywiki.js
NODE_ARGS=--server
echo "Starting $NODE_EXEC $APP_DIR/$NODE_APP $NODE_ARGS"
  cd $APP_DIR
  exec                             \
  setuidgid $USER                  \
  $NODE_EXEC $NODE_APP $NODE_ARGS
' > /var/service/tiddlywiki/run

    #
    # Create log/run
    #
echo '#!/bin/sh
#
exec                                    \
    /usr/local/bin/setuidgid smelog     \
    /usr/local/bin/multilog t s5000000  \
    /var/log/tiddlywiki' > /var/service/tiddlywiki/log/run

start the service

sv u tiddlywiki

check the log files to see if it worked

tail /var/log/tiddlywiki/current

Create init.d script and startup.shutdown scripts

This segment of code will create the scripts needed to start the service at boot and to stop the service at shutdown.

SERVICE=tiddlywiki
#
cd /etc/rc.d/init.d
ln -s daemontools $SERVICE
cd /etc/rc.d/rc0.d
ln -s /etc/rc.d/init.d/e-smith-service K01$SERVICE
cd /etc/rc.d/rc1.d
ln -s /etc/rc.d/init.d/e-smith-service K01$SERVICE
cd /etc/rc.d/rc6.d
ln -s /etc/rc.d/init.d/e-smith-service K01$SERVICE
cd /etc/rc.d/rc7.d
ln -s /etc/rc.d/init.d/e-smith/service S99$SERVICE

Create config db entry

/etc/rc.d/init.d/e-smith-service will start a service whose status is enabled, and will not start it otherwise.

SERVICE=tiddlywiki
config set $SERVICE service access public status enabled

Proxypass Domain for WAN access

I found that I needed to proxypass a domain. An alias/directory/location proxypass generated errors and prevented edits from saving correctly.

DOMAIN=tiddlywiki.domain.tld
db domains set $DOMAIN domain Nameservers internet ProxyPassTarget http://localhost:8080/ TemplatePath ProxyPassVirtualHosts
signal-event domain-create $DOMAIN

Update to the latest tiddlywiki code

cd /opt/tiddlywiki && setuidgid www npm update tiddlywiki && sv t tiddlywiki

COMPLETE Removal

DOMAIN=tiddlywiki.domain.tld
signal-event domain-delete $DOMAIN
db domains delete $DOMAIN
#
SERVICE=tiddlywiki
config delete $SERVICE
find /etc/rc.d -name "*$SERVICE*" -exec 'rm' -f "{}" \;
'rm' -rf /service/$SERVICE
'rm' -rf /var/service/$SERVICE
'rm' -rf /var/log/$SERVICE
#
cd /opt/$SERVICE
npm remove $SERVICE
cd /opt
'rm' -rf /opt/$SERVICE

Notes on check_earlytalker

Why did you remove the Request_for_deletion template on the check_earlytalker page? AFAIK it is obsolete and should be deleted according to http://forums.contribs.org/index.php/topic,46234.msg226418.html#msg226418 - — Cactus (talk | contribs) 07:05, 16 February 2012 (MST)

Here's my understanding of the Request_for_deletion addition to this page:

1) Piran posted a link to this page in a thread with the text "Install the check_earlytalker plugin": http://forums.contribs.org/index.php/topic,46229.msg226377.html#msg226377

2) Charlie noticed Piran's post, and assumed the wiki page described how to install check_earlytalker, so he made his post that the page is obsolete: http://forums.contribs.org/index.php/topic,46234.msg226418.html#msg226418

3) The page was updated with the 'Request_for_deletion' template

4) I added the reasoning to the 'talk' page outlining why the page is NOT obsolete (see below)

5) Over a year later, I removed the 'Request_for_deletion' template, assuming everyone had read and agreed with the reasoning I outlined on the 'talk' page.

Copied from http://wiki.contribs.org/Talk:Qpsmtpd_check_earlytalker - Unless there is more information elsewhere, I don't feel that this page is obsolete.

Despite the language used by piran in his mention of this page at http://forums.contribs.org/index.php/topic,46229.msg226377.html#msg226377, this page is not about adding check_earlytalker - which is included by default as mentioned by Charlie in the forum post referenced above.

This page is about:

   Documenting the functionality of check_earlytalker (what it does and why it does it)
   Documenting how to change the timeout value applied by check_earlytalker
   Documenting how to monitor check_earlytalker to see if it is being used to deny email

Install mariadb 'alongside' mysql

Installation

Create service account

Download MariaDB and extract into /opt

Create folders and links

Setup my.cnf and init.d

file and folder ownership

Initialize Environment

Start Server

set mariadb root password

Testing

Configure to start at boot

Raw notes

Gotchas

my.cnf

/var/run/mariadb

/etc/rc.d/init.d/mariadb

Backup & Restore

/etc/e-smith/events/actions/mariadb-dump-tables

/etc/e-smith/events/actions/mariadb-load-tables

Automation

Backup

Restore - NEEDS WORK

Install Moodle 2.6 using git

Requirements

DB Version

OPcache

Installation

Prepare your server

Install useful php modules

Create an ibay

Create a database

Install Moodle

Download & Checkout using git

Run the moodle installer

Correct database settings if necessary

Optional Settings

Scan Uploads using ClamAV

Authentication Settings

Create Additional Administrators

TiddlyWiki5 Using Node.js

Prerequisites

Assumptions

Installation

Create daemontools scripts, folders, etc

start the service

check the log files to see if it worked

Create init.d script and startup.shutdown scripts

Create config db entry

Proxypass Domain for WAN access

Update to the latest tiddlywiki code

COMPLETE Removal

Notes on check_earlytalker

Navigation menu

Search