Version 8.0 testing

This document lists tests that we need to do to get version 8.0 closer to a final release. It's an attempt to invite as many testers as possible and to coordinate our testing work.

The document should be used as Work In Progress so please feel free to add additional test scenarios and comments

All tests here should be based on the latest available version '8.0 beta 7'. (with updates including smeupdates-testing you have a preview of a work in progress '8.0RC1')

In general:

check the related logfile(s) for each test
don't hesitate to raise a bug if there is an issue or something suspicious but check the Bugtracker before whether this might already be reported
A list of currently reported bugs for SME 8.0 is here: bugzilla matrix

Test guidelines

The purpose of the page is mostly to verify features are working, you can list a bug number if you like but we don't intend to duplicate the tracking provided by bugzilla:

Executive Summary

In general everything works as well as in SME 7, if not better.

Verification

Create a test environment, either virtually or with spare hardware. Install the latest release, add some test users/ibays/domains/websites, send some internal e-mails, copy some files to/from ibays, perform a backup and restore. Perform as many functions that you can based on your knowledge and test environment abilities. Play and have fun :)

The scenario's should be as complex as needed, but also as simple as possible, testing one unique feature where possible.

For instance creating an ibay could mean something like this, perhaps even incomplete as I am doing this from the top of my head:

log in in server-manager
create ibay
test for directory created (-f /home/e-smith/ibays/ibayname)
check for account (ibayname) created
check for group (ibayname) created
analyze log for errors/exceptions

but also (and that is the most work mostly) test exceptions (names too long, starting with a number, containing a _, as well as all other exceptions):

log in in server-manager
create ibay with invalid name
check the error in the server-manager screen
check no account (ibayname) is created
check no group (ibayname) is created
check that the error is present in the log file

Test Schedule and Header codes

A sample test sheet is provided below. Check one feature at a time and add comments. We will use a shorthand in the header line so we can check progress quickly

header blank - unchecked
header ? - questions, or more to do
header bug - there is a bug
header Pass - means the section has been tested and has passed

Installation

Installation types

Different installation scenarios re. software RAID, LVM and NOSPARE usage, etc. Note that you need to test each scenario separately, meaning reinstalling from scratch for each test - you cannot modify a Raid configuration after a system has been installed.

to check: /var/log/messages; server-console, item 5; cat /proc/mdstat (which is the same as server-console, item 5;)

RAID1 nolvm spares=2 - Pass

Comments:

RAID1 "nolvm" - Pass

Comments: Installation was RAID1, nolvm with one physical disk only; adding second disk still to be tested

RAID5 (with lvm) spares=0 - Pass

Comments:

RAID5 nolvm spares=2 - Pass

Comments:

RAID6 (with lvm) spare=1 multipart - Pass

Comments:

RAIDx "nolvm"

Comments:

Upgrading

Upgrade from older versions

From 7.4

CD upgrade from a clean SME7.4 installation to SME8b4.

The message as below in /var/log/messages can be safely ignored, see: ~~bugzilla:5486~~:

 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1060 (42S21) at line 75: Duplicate column name 'File_priv'
 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1060 (42S21) at line 81: Duplicate column name 'Grant_priv'
 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1060 (42S21) at line 82: Duplicate column name 'Grant_priv'
 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1060 (42S21) at line 83: Duplicate column name 'Grant_priv'
 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1060 (42S21) at line 94: Duplicate column name 'ssl_type'
 Sep 24 00:22:18 sme74clean mysql.init: ERROR 1061 (42000) at line 105: Duplicate key name 'Grantor'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1054 (42S22) at line 131: Unknown column 'Type' in 'columns_priv'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1060 (42S21) at line 153: Duplicate column name 'type'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1060 (42S21) at line 163: Duplicate column name 'Show_db_priv'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1060 (42S21) at line 180: Duplicate column name 'max_questions'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1060 (42S21) at line 190: Duplicate column name 'Create_tmp_table_priv'
 Sep 24 00:22:19 sme74clean mysql.init: ERROR 1060 (42S21) at line 193: Duplicate column name 'Create_tmp_table_priv'
 Sep 24 00:22:20 sme74clean mysql.init: ERROR 1054 (42S22) at line 282: Unknown column 'Create_view_priv' in 'where clause'
 Sep 24 00:22:22 sme74clean mysql.init: ERROR 1054 (42S22) at line 317: Unknown column 'Create_routine_priv' in 'where clause'
 Sep 24 00:22:24 sme74clean mysql.init: ERROR 1054 (42S22) at line 366: Unknown column 'Create_user_priv' in 'where clause'
 Sep 24 00:22:25 sme74clean mysql.init: ERROR 1060 (42S21) at line 386: Duplicate column name 'Routine_type'

/root/upgrade.log :

For db4 error(22): (check if it has been fixed) ~~bugzilla:5489~~
For No module mptscsi found: ~~bugzilla:5490~~
For PHP Warning: PHP Startup: Unable to load dynamic library: ~~bugzilla:5491~~

from 7.4.0 and lower

If in doubt, first update to 7.5.1

Console

cosmetic bug 3561 (check if it has been fixed) ~~bugzilla:3561~~

briefly displays 'Could not open pci table...'

Check Status

Pass - the server displays 'This sever has been running for ...'

Configure Server

Basic configuration works, eg NIC's, password, host and domain names

DHCP server

test IP allocation - Pass

for predefined MAC addresses (= 'static' IP allocation)

for dynamic IPs

Server-gateway

Pass

Comments: Running as gateway behind broadband router with static IP for several weeks; all sorts of incoming connections tested (email, ssh, webmail, server-manager access) successfully, also outgoing traffic from various PCs (Windows, Linux) on LAN

Server-only

Comments:

Test internet

Pass - Prompt displays, then 'The test was successful...'

Reboot

Pass for each of

reboot

reconfigure

shutdown

Raid

pass

console, option 5, displays correctly for single disk

displays correctly for various RAID scenarios, all those combinations which we tested under SME8.0_QA#Installation_types

another disk added to a one disk system, open console, option 5, prompted to add drive, drive added, option 5 states all drives in clean state.

Access Server Manager

Pass - server-manager displays and navigation works

View Support

Pass, The GPL text displays

Backup to USB

console restore, pass

console backup, pass

Server Manager ?

Collaboration ?

Users

create new user account - PASS
reset user password - Pass

Comments: server-manager message OK, eventlog check OK

lock/unlock account - Pass

Comments: both activities checked; server-manager messages OK, eventlog check OK

add/remove user to different groups - Pass

Comments: added user to various other groups and later removed from all of these groups

delete user account - Pass
re create deleted user account - PASS
reset user password user account - Pass

Groups

Add, modify and remove groups - PASS

Quotas

Setting quota (soft:10MB / hard: 20MB) and going over hard limit works: -PASS

 # dd if=/dev/zero of=/mnt/user-a/user-a_21MB bs=22020096 count=1
 1+0 records in
 1+0 records out
 dd: closing output file `/mnt/user-a/user-a_21MB': No space left on device

Setting quota (soft:20MB / hard: 40MB) and going over soft limit works: -PASS

 To: User B <user-b@testdomain.lan>
 From: "Automated quota report" <do-not-reply@testdomain.lan>
 Subject: You have exceeded your disk quota
 
 Your current disk usage: 21.08 Mb
 Your maximum usage: 40.00 Mb
 Warnings start at: 20.00 Mb
 Grace period ends: Fri Oct  2 09:06:22 2009
 System name: sme8b4clean

Pseudonyms

Pass

created a pseudonym jane to sent to usera

created a pseudonym jane@test.org to userb

sent mail to both which was received as expected

Information Bays x

User access via file sharing

basic smb file access working as expected. Permissions checked (smb) with several groups, users and all permission-combinations

Public access via web ?

Primary website access tested form public Internet. If not configured, the default (".. under construction ..") works.
Ibay accessible for http(s) from outside network without password - PASS
Ibay accessible for http(s) from outside network with a password -NEEDS TESTING
Ibay accessible for http(s) from local network without password and outside network with a pass -NEEDS TESTING

Local access via web

Primary website accessible for http(s) from the local network -PASS

Ibay not accessible for http(s) from local network when set to 'No Access' -PASS
Ibay accessible for http(s) from local network without password -PASS
Ibay accessible for http(s) from local network with a password -PASS

/var/log/httpd/error_log (Bug #5492):

 [Thu Sep 24 22:45:56 2009] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
 [Thu Sep 24 22:45:56 2009] [warn] RSA server certificate CommonName (CN) `sme8b4clean.testdomain.lan' does NOT match server name!?

Public access via ftp -x

Local access via ftp

Primary & ibay not accessible when ftp set to 'No Access' -PASS
Primary ibay accessible anonymous for download, not upload -PASS
Ibay (no password required) accessible anonymous without password for download, not upload -PASS
Ibay (password required) accessible anonymous only with password for download, not upload -PASS

User homedir accessible with username+pass for download and upload -PASS
Ibay accessible with username+pass for download and upload (with ibay permissions correctly set) -PASS
Ibay accessible with username+pass for download and not upload (with ibay permissions correctly set) -PASS
Ibay not accessible with username+pass (with ibay permissions correctly set) -PASS

Dynamic content ?

PHP scripts in Primary ibay are interpreted and displayed -PASS
PHP scripts in ibay are/not interpreted and displayed when dynamic content is enabled/disabled -PASS

CGI scripts (cgi-bin) -NEEDS TESTING

Administration x

Backup and restore

Backup

to desktop

a server with a small amount of data created a smeserver.tgz correctly

to tape x

TO DO

to workstation

pass - tested a set of three to a cifs share

to usb disk

pass

Restore

pass

from desktop

Restoring from desktop is actually via the console, put you .tgz on a cd or usb device, pass

from tape x

TO DO

from workstation

pass - tested a set of three to a cifs share, with verification

from usb disk

pass

View log files

view log files works, including filtering and highlighting

download logfile works

Mail log files

local mail delivery is reported

external mail delivery is tested (ongoing use for a 'production' home server) and email logs are OK

Reboot and shutdown

Pass for each of

reboot

reconfigure

shutdown

Security x

Remote access

VPN / PPTP

Pass

Set a number of PPTP clients in server manager and test remote access (repeatedly)

Remote management

SSH

Pass - remote access from local network on port other than 22
External user/password access denied (port 22 and other) when set to 'No access' or 'Local network' --PASS
External user/password access allowed (port 22 and other) when set to 'Entire internet' --PASS
External user/password access denied (port 22 and other) for root when 'allow administrative commandline' to 'No' --PASS
External user/password access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'No' --PASS
External user/password access denied (port 22 and other) when 'allow standard password' set to 'No' -PASS
External certificate access denied (port 22 and other) for root when 'administrative commandline' to 'No' and 'allow standard password' set to 'No' -PASS
External certificate access allowed (port 22 and other) for root when 'administrative commandline' to 'Yes' and 'allow standard password' set to 'No' -PASS
External certificate access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'No' and 'allow standard password' set to 'No' -PASS
External certificate access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'Yes' and 'allow standard password' set to 'No' -PASS

FTP

Ftp access allowed for Primary, ibays and users when access set to 'Allowe access ...' -PASS
Ftp access refused for Primary, ibays and users when access set to 'No Access' -PASS

Local Networks x

Port forwarding

server gateway - PASS

Tested successfully with port 3389 (RDP) to WIndows terminal server on LAN

server only - pass

"This server is currently in serveronly mode and portforwarding is not possible."

Proxy settings

Mail

Smtp proxy enabled - PASS

Test with Thunderbird client - Email delivery works and the headers clearly show that the local smtp server has been used for the delivery rather the smtp that was setup in the email client

Smtp proxy disabled - PASS

Follow up test to above with the same Thunderbird configuration - this time the intended, external smtp server has been used for the delivery and NOT the local smtp server.

Web

net access via the squid proxy works

net access is disabled via squid proxy if disabled

Configuration

Software Installer

Updates

updates via the panel work

the updates available warning displays

Settings

Settings can be changed

Installing additional software works

Installing:
 wvdial               i386         1.54.0-5.2.2.1            base         131 k
Installing for dependencies:
 libwvstreams         i386         4.2.2-2.1                 base         657 k

and Removing

Removing:
 wvdial          i386          1.54.0-5.2.2.1          installed          264 k

Date and time

'Set Date and Time' manually works

'Configure Network Time Server' works, if you enter a valid time server

Workgroup

Domain controller

Now also working for windows 7 PC's

PDC - setup Primary Domain Controller - Pass

Steps:

join PC to domain (as user 'admin') and test domain user logon on that PC - Pass
join (another) PC to domain (as a different user from group 'Domain Admins', NOT user admin) - Pass

Comments: The user 'michael' is part of that group:

db accounts show domain-admin-group 
domain-admin-group=group
   Description=Domain Admins
   Gid=5010
   Members=admin,michael
   Uid=5010

Comments: Here is a log extract from /var/log/messages for the latest join:

Aug 30 07:04:52 mdo002sv esmith::event[20711]: Running event handler: /etc/e-smith/events/machine-account-create/S10create-machine-account 
Aug 30 07:04:52 mdo002sv /etc/e-smith/events/machine-account-create/S10create-machine-account[20712]: /home/e-smith/db/accounts: OLD twt042lt$ (undefined) 
Aug 30 07:04:52 mdo002sv /etc/e-smith/events/machine-account-create/S10create-machine-account[20712]: /home/e-smith/db/accounts: NEW twt042lt$=machine 
Aug 30 07:04:52 mdo002sv esmith::event[20711]: create-machine-account twt042lt$: Creating Unix user and group  
Aug 30 07:04:52 mdo002sv esmith::event[20711]: create-machine-account twt042lt$: Locking account  
Aug 30 07:04:52 mdo002sv esmith::event[20711]: Locking password for user twt042lt$.  
Aug 30 07:04:52 mdo002sv esmith::event[20711]: passwd: Success  
Aug 30 07:04:52 mdo002sv esmith::event[20711]: S10create-machine-account=action|Event|machine-account-create|Action|S10create-machine-account|Start|1251572692 236608|End|1251572692 708624|Elapsed|0.472016

domain user password change via Ctrl-Alt-Del - Partially

Comments: Several combinations tested, incl. invalid combinations (e.g. password too short); successfully changed; case sensitivity tested; testing was done with the following setting in place:

db configuration show passwordstrength
passwordstrength=configuration
   Admin=none
   Ibays=none
   Users=none

Might need more testing with better password strength.

Workgroup server

Connect to samba share, pass

Roaming profiles

Tested successfully with WinXP Pro AND Win7 clients - PASS

Printers

Attached/created LTP,USB and network printer; Printing test page:

USB printer -PASS
LTP printer -PASS
Network printer -PASS

Hostnames

set a hostname to a remote server works

set a hostname to self works

set hostname for other host on the local network - passed (tested with several other hosts)

Domains ?

2nd domain added, web and email works

DNS options to be tested

Email ?

Access ?

webmail works

pop3(s)

Comments:

imap(s) - Pass

Comments: Intensive, ongoing testing with imap & imaps access from LAN PCs and external

to check: /var/log/pop3(s)/current; /var/log/imap(s)/current

Filtering x

Reception x

Delivery x

Antivirus

Clam/email integration - Pass

to check: /var/log/qmail/current; /var/log/spamd/current; /var/log/qpsmtpd/current; /var/log/clamd/current; /var/log/clamd/smeserver-clamscan.log; email headers

Comments: Server runs in a semi production state - as a home server with various email accounts for family members (static IP, registered domain name). Have done regular checks for quite some time and there is nothing unusual or unexpected in all email logs. Incl. email header checks for spam recognition and filtering.

regular Clam scans - Pass

enable regular scanning in server-manager and check results (email to admin)

Comments: Runs weekly without problems, emails to admin working.

Review

Basic server information is displayed

Miscellaneous

Support

The GPL text is displayed

Starter Website

Site created after entering text as prompted

SME8.0 QA

Version 8.0 testing

Test guidelines

Executive Summary

Verification

Test Schedule and Header codes

Installation

Installation types

Upgrading

Upgrade from older versions

From 7.4

from 7.4.0 and lower

Console

Check Status

Configure Server

DHCP server

Server-gateway

Server-only

Test internet

Reboot

Raid

Access Server Manager

View Support

Backup to USB

Server Manager ?

Collaboration ?

Users

Groups

Quotas

Pseudonyms

Information Bays x

User access via file sharing

Public access via web ?

Local access via web

Public access via ftp -x

Local access via ftp

Dynamic content ?

Administration x

Backup and restore

Backup

to desktop

to tape x

to workstation

to usb disk

Restore

from desktop

from tape x

from workstation

from usb disk

View log files

Mail log files

Reboot and shutdown

Security x

Remote access

VPN / PPTP

Remote management

SSH

FTP

Local Networks x

Port forwarding

Proxy settings

Mail

Web

Configuration

Software Installer

Updates

Settings

Date and time

Workgroup

Domain controller

Workgroup server

Roaming profiles

Directory

Printers

Hostnames

Domains ?

Email ?

Access ?

Filtering x

Reception x