Version 9.# testing
This document lists tests that we need to do to get version 9.# to a final release. It's an attempt to invite as many testers as possible and to coordinate our testing work.
The document should be used as Work In Progress so please feel free to add additional test scenarios and comments.
All tests here should be based on the latest available version '9.# Beta X'. with updates including smeupdates-testing you have a preview of a work in progress
- check the related logfile(s) for each test
- don't hesitate to raise a bug if there is an issue or something suspicious but check the Bugtracker before whether this might already be reported
- A list of currently reported bugs for SME Server 9.# is here: 
The purpose of the page is mostly to verify features are working, you can list a bug number if you like but we don't intend to duplicate the tracking provided by bugzilla:
There has not been enough testing to indicate how well SME Server 9.# is working.
Create a test environment, either virtually or with spare hardware. Install the latest release, add some test users/ibays/domains/websites, send some internal e-mails, copy some files to/from ibays, perform a backup and restore. Perform as many functions that you can based on your knowledge and test environment abilities. Play and have fun :)
The scenario's should be as complex as needed, but also as simple as possible, testing one unique feature where possible.
For instance creating an ibay could mean something like this, perhaps even incomplete as I am doing this from the top of my head:
- log in in server-manager
- create ibay
- test for directory created (-f /home/e-smith/ibays/ibayname)
- check for account (ibayname) created
- check for group (ibayname) created
- analyze log for errors/exceptions
but also (and that is the most work mostly) test exceptions (names too long, starting with a number, containing a _, as well as all other exceptions):
- log in in server-manager
- create ibay with invalid name
- check the error in the server-manager screen
- check no account (ibayname) is created
- check no group (ibayname) is created
- check that the error is present in the log file
Test Schedule and Header codes
A sample test schedule is provided below. Check one feature at a time and add comments. We will use a shorthand in the header line so we can check progress quickly. The test list below starts with no headers since we start afresh. The intention is for testers to update this document to reflect their testing, thus avoiding duplication.
- header blank = no header - unchecked
- header ? - questions, or more to do [please provide your name for follow-up]
- header BUG - there is a bug - provide Bug reference number, i.e [Bug xxxx]
- header PASS - means the section has been tested and has passed [please provide your name for follow-up]
Some sample comments have been left from testing previous SME Server versions for information since they outline useful procedures, please add your own comments as applicable.
WARNING!!! There are no cmdline options that are supported yet (for SME 9), refer
Different installation scenarios re. software RAID, LVM and NOSPARE usage, etc. Note that you need to test each scenario separately, meaning reinstalling from scratch for each test - you cannot modify a Raid configuration after a system has been installed.
Check this for each configuration:
- server-console, item 5
- cat /proc/mdstat (which is the same as server-console, item 5;)
- RAID1 nodmraid
Comment: the option "nodmraid" seems to work (dz00te)
- RAID1 raid=1
Comments: the raid=1 options doesn't force the raid1. see
- RAID1 nolvm spares=2
- RAID1 nolvm
Comments: Installation was RAID1, nolvm with one physical disk only; adding second disk still to be tested. BUG [check Bug #7528]
- RAID5 (with lvm) spares=0
- RAID5 nolvm spares=2
- RAID6 (with lvm) spare=1 multipart
- RAIDx "nolvm"
In-place upgrades are not supported from SME8.#. It is necessary to backup and then restore. /boot partition is always RAID 1.
In-place upgrades to SME 9.1" *are* supported - from 9.0 only.
Upgrade from older versions
In-place upgrades to SME 9.1" *are* supported - from 9.0 only.
- The server displays 'This server has been running for ...'[PASS]
Comment: Tested on beta 3 [chris burnat]
- Basic configuration works?, eg NIC's, password, host and domain names [PASS] Comment: Tested on beta 3 [chris burnat]
- test IP allocation
- for predefined MAC addresses (= 'static' IP allocation) [PASS] (chrisb)
- for dynamic IPs
- Check functionality [PASS] Gateweay behing ADSL router, static IP, 5 WStations on LAN. Tested ssh,webmail, server-manager access from remote, FTP, etc - WS seems happy, Ubuntu MAC and Windows XP and 7Pro. (chrisb)
Sample Comments: Running as gateway behind broadband router with static IP for several weeks; all sorts of incoming connections tested (email, ssh, webmail, server-manager access) successfully, also outgoing traffic from various PCs (Windows, Linux) on LAN
- Check functionality
- Private server & gateway
- Prompt displays, then 'The test was successful...' [PASS] Comment: Tested on beta 3 [chris burnat]
- Check each of the console options: (david beveridge)
- reboot [PASS]
- reconfigure [PASS]
- shutdown [PASS]
- Check each of the following:
- console, option 5, displays correctly for single disk [PASS] Comment: Tested on beta 3 [chris burnat]
- console, option 5, displays correctly for various RAID scenarios, all those combinations which we tested under SME8.0_QA#Installation_types
- another disk added to a one disk system, open console, option 5, prompted to add drive, drive added, option 5 states all drives in clean state [FAIL].
Comment: Does not seem to work, installed on 200GB, then added another drive (500GB), item 5 does not detect the new drive. Repeated this with sme8, drive is recognised and ready to add to mirror. (Bug #7585 chrisb)
Access Server Manager
- server-manager displays and navigation works [PASS] (chrisb)
Comments: Some problems reported with access for alpha3 - check [Bug #7534] and [Bug #7538]
- The GPL text displays [PASS] (chris burnat)
Backup to USB
- Check each of the following:
- console restore [FAILED] Comment: performed restore from sme8 test system during installation of 9b3 - appears to restore OK, however, cannot access server-manager as reported by Jason [BUG #7547] However, restore of sm9 was also tested and was ok (chrisb)
Additional comments: whilst restore is completed successfully, there are a number of unnecessary lines of text appearing on the screen, refer [Bug #7539] and [Bug #7540]
- console backup [PASS] Done one backup to USB and restore = ok.
- Accessing server-manager (local) - [PASS] (Chris burnat)
- Accessing server-manager (from remote location) - [PASS] (Chris burnat)
- create new user account
- reset user password
Comments: i.e. server-manager message OK, eventlog check OK
- lock/unlock account
Comments: i.e. both activities checked; server-manager messages OK, eventlog check OK
- add/remove user to different groups
Comments: i.e. added user to various other groups and later removed from all of these groups
- delete user account
- re create deleted user account
- reset user password user account
- Add, modify and remove groups
- Setting quota (soft:10MB / hard: 20MB) and going over hard limit works:
# dd if=/dev/zero of=/mnt/user-a/user-a_21MB bs=22020096 count=1 1+0 records in 1+0 records out dd: closing output file `/mnt/user-a/user-a_21MB': No space left on device
- Setting quota (soft:20MB / hard: 40MB) and going over soft limit works:
To: User B <email@example.com> From: "Automated quota report" <firstname.lastname@example.org> Subject: You have exceeded your disk quota Your current disk usage: 21.08 Mb Your maximum usage: 40.00 Mb Warnings start at: 20.00 Mb Grace period ends: Fri Oct 2 09:06:22 2009 System name: sme8b4clean
- Test the following:
- created a pseudonym jane to sent to usera
- created a pseudonym email@example.com to userb
- Comment: i.e. sent mail to both which was received as expected
User access via file sharing
- basic smb file access working as expected
- Permissions checked (smb) with several groups, users and all permission-combinations
Public access via web
- Primary website access tested form public Internet. If not configured, the default (".. under construction ..") works
- Ibay accessible for http(s) from outside network without password
- Ibay accessible for http(s) from outside network with a password
- Ibay accessible for http(s) from local network without password and outside network with a password
Local access via web
- Primary website accessible for http(s) from the local network
- Ibay not accessible for http(s) from local network when set to 'No Access'
- Ibay accessible for http(s) from local network without password
- Ibay accessible for http(s) from local network with a password
/var/log/httpd/error_log (Bug #5492): (still existing?)
[Thu Sep 24 22:45:56 2009] [warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?) [Thu Sep 24 22:45:56 2009] [warn] RSA server certificate CommonName (CN) `sme8b4clean.testdomain.lan' does NOT match server name!?
Public access via ftp
- check functionality as applicable
Local access via ftp
- Primary & ibay not accessible when ftp set to 'No Access'
- Primary ibay accessible anonymous for download, not upload
- Ibay (no password required) accessible anonymous without password for download, not upload
- Ibay (password required) accessible anonymous only with password for download, not upload
- User homedir accessible with username+pass for download and upload
- Ibay accessible with username+pass for download and upload (with ibay permissions correctly set)
- Ibay accessible with username+pass for download and not upload (with ibay permissions correctly set)
- Ibay not accessible with username+pass (with ibay permissions correctly set)
- PHP scripts in Primary ibay are interpreted and displayed
- PHP scripts in ibay are/not interpreted and displayed when dynamic content is enabled/disabled
- CGI scripts (cgi-bin)
Backup and restore
=====to desktop===== [PASS] Comment: backup OK, yet to test restore (chrisb) Comment: i.e. a server with a small amount of data created a smeserver.tgz correctly
=====to workstation===== [PASS]
Comment: i.e. tested a set of three to a cifs share
=====to usb disk ===== [PASS] Tested backup and restore, seems OK. Comment:
Comment: =====from desktop===== [PASS] Comment: i.e. restoring from desktop is actually via the console, put you .tgz on a cd or usb device, pass
=====from workstation===== [PASS]
Comment: i.e. tested a set of three to a cifs share, with verification
=====from usb disk ===== [PASS]
Comment: Restore from sme9 backup is ok, however, restore from sme8 creates Havoc [Bug 7547]
View log files
- view log files works, including filtering and highlighting
- download logfile works
Mail log files
- local mail delivery is reported
- external mail delivery is tested (ongoing use for a 'production' home server) and email logs are OK
Reboot and shutdown
- Test from server-manager for each of:
- reboot [PASS] (chris burnat)
- reconfigure [PASS] (chris burnat)
- shutdown [PASS] (chris burnat)
VPN / PPTP
- Check functionality
- Set a number of PPTP clients in server manager and test remote access (repeatedly)
Submitted bug 7546 because PPTP login fails due to missing radiusclient rpm. (david beveridge)
Check the following:
- remote access from local network on port other than 22 [PASS chrisb]
- External user/password access denied (port 22 and other) when set to 'No access' or 'Local network'
- External user/password access allowed (port 22 and other) when set to 'Entire internet'
- External user/password access denied (port 22 and other) for root when 'allow administrative commandline' to 'No'
- External user/password access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'No'
- External user/password access denied (port 22 and other) when 'allow standard password' set to 'No'
- External certificate access denied (port 22 and other) for root when 'administrative commandline' to 'No' and 'allow standard password' set to 'No'
- External certificate access allowed (port 22 and other) for root when 'administrative commandline' to 'Yes' and 'allow standard password' set to 'No'
- External certificate access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'No' and 'allow standard password' set to 'No'
- External certificate access allowed (port 22 and other) for user (if configured) when 'administrative commandline' to 'Yes' and 'allow standard password' set to 'No'
- Ftp access allowed for Primary, ibays and users when access set to 'Allowe access ...'
- Ftp access refused for Primary, ibays and users when access set to 'No Access'
Check the following scenarios:
- server gateway
Comment: i.e. tested successfully with port 3389 (RDP) to WIndows terminal server on LAN
- server only
Comment: i.e."This server is currently in server-only mode and port forwarding is not possible."
- Smtp proxy enabled
Comment: i.e. Test with Thunderbird client - Email delivery works and the headers clearly show that the local smtp server has been used for the delivery rather the smtp that was setup in the email client
- Smtp proxy disabled
Comment: i.e. Follow up test to above with the same Thunderbird configuration - this time the intended, external smtp server has been used for the delivery and NOT the local smtp server.
- Smtp proxy blocked (recently implemented)
Check the following:
- net access via the squid proxy works
- net access is disabled via squid proxy if disabled
- updates via the panel work [PASS] (David Beveridge)
- the updates available warning displays on this page [PASS] (David Beveridge)
- Settings can be changed
- Installing additional software works
Installing: wvdial i386 1.54.0-220.127.116.11 base 131 k Installing for dependencies: libwvstreams i386 4.2.2-2.1 base 657 k
Removing: wvdial i386 1.54.0-18.104.22.168 installed 264 k
Date and time
'Set Date and Time' manually works [PASS] (David Beveridge)
'Configure Network Time Server' works if you enter a valid time server [PASS] (David Beveridge)
Now also working for windows 7 PC's
- PDC - setup Primary Domain Controller
- join PC to domain (as user 'admin') and test domain user logon on that PC
- join (another) PC to domain (as a different user from group 'Domain Admins', NOT user admin)
Comments: The user 'michael' is part of that group:
db accounts show domain-admin-group domain-admin-group=group Description=Domain Admins Gid=5010 Members=admin,michael Uid=5010
Comments: i.e. log extract from /var/log/messages for the latest join:
Aug 30 07:04:52 mdo002sv esmith::event: Running event handler: /etc/e-smith/events/machine-account-create/S10create-machine-account Aug 30 07:04:52 mdo002sv /etc/e-smith/events/machine-account-create/S10create-machine-account: /home/e-smith/db/accounts: OLD twt042lt$ (undefined) Aug 30 07:04:52 mdo002sv /etc/e-smith/events/machine-account-create/S10create-machine-account: /home/e-smith/db/accounts: NEW twt042lt$=machine Aug 30 07:04:52 mdo002sv esmith::event: create-machine-account twt042lt$: Creating Unix user and group Aug 30 07:04:52 mdo002sv esmith::event: create-machine-account twt042lt$: Locking account Aug 30 07:04:52 mdo002sv esmith::event: Locking password for user twt042lt$. Aug 30 07:04:52 mdo002sv esmith::event: passwd: Success Aug 30 07:04:52 mdo002sv esmith::event: S10create-machine-account=action|Event|machine-account-create|Action|S10create-machine-account|Start|1251572692 236608|End|1251572692 708624|Elapsed|0.472016
- domain user password change via Ctrl-Alt-Del
Comments: i.e. several combinations tested, incl. invalid combinations (e.g. password too short); successfully changed; case sensitivity tested; testing was done with the following setting in place:
db configuration show passwordstrength passwordstrength=configuration Admin=none Ibays=none Users=none
Might need more testing with better password strength.
- Connect to samba share
Testing from Windows 2008R2 Server with SME IP Address entered into WINS on IPv4 Config. WorkGroupNames Matching
Also tried from Windows 8 Pro Workstation on a remote subnet (listed as a local network). Different WorkGroup Name, WINS address set to SME Server.
net use x: \\sme9a3\Primary /user:admin
works and I have write access; login as a user and has read-only
create an ibay and grant rights then user can write to the volume
net view shows my printer as well. [PASS]
- Test with WinXP Pro AND Win7 Pro clients
Check the following:
- Directory (LDAP) access
- Test access from various email clients, list client, platform, Operating system, local and or external:
- Webmail (list address book contents via string like "mich" shows all Michael(s))
- Thunderbird LDAP test on local network
(David Beveridge) Tested Thunderbird, x86_64, CentOS 6.4 to a local network at another site. [PASS] Tested Webmail. LDAP contacts appear in there too [PASS]
Attached/created LTP,USB and network printer; Printing test page:
- USB printer
- LTP printer
- Network printer
set a hostname to a remote server works
set a hostname to self works
set hostname for other host on the local network - passed (tested with several other hosts)
- 2nd domain added, web and email works
- Test all DNS options
Test functionality for the following:
- webmail [PASS] (chris burnat)
Seems to be working ok. Send and received. [PASS] I sent a message to a bad email with UTF-8 encoding. When the bounce came back the UTF-8 sections were all messed up. I'm talking about Chinese and Cyrillic etc. I dunno if this is a problem. I had horde set to English (American). The copy I received in GMail was perfect.(David Beveridge)
- pop3(s) [FAIL]
Comments: pop3 ok on port 110, but fails on port 995 (chris burnat)
Used Sending and Receiving with Thunderbird with no problems on CentOS 6.4 x86_64 workstation. [PASS]
Windows 8 Pro Metro Mail Application Cannot setup the account.
Basically SME Server won't allow plain text mode (despite giving the appearance that it can from Server-manager) and if you try to set up the Windows client in Secure Mode it complains the the certificates need to be installed by the administrator. I assume this is because they are a self signed cert. Not really a problem as certs are only about $10, But I cannot see where to install it in the SME manager. So as I see it the bug is that SME server offers plain text in the manager interface but the back end doesn't actually support it. [FAIL]
Bug 7549is now fixed. Can this be retried.
- check: /var/log/pop3(s)/current -nothing unusual (chrisb)
- check: /var/log/imap(s)/current -nothing unusual (chrisb)
- Check: /var/log/qmail/current
- Check: /var/log/spamd/current
- Check: /var/log/qpsmtpd/current
- Check: /var/log/clamd/current
- Check: /var/log/clamd/smeserver-clamscan.log
- Check: email headers
Comments: i.e. Server runs in a semi production state - as a home server with various email accounts for family members (static IP, registered domain name). Have done regular checks for quite some time and there is nothing unusual or unexpected in all email logs. Incl. email header checks for spam recognition and filtering.
- Check regular Clam scans
- Enable regular scanning in server-manager and check results (email to admin)
Comments: i.e. Runs weekly without problems, emails to admin working.
Basic server information is displayed [PASS] (chris burnat)
The GPL text is displayed (David Beveridge) [PASS]
Site created after entering text as prompted
That pretty much worked fine, so long as you're using an ANSI character set.
Might have to do a NFR to fix that. [PASS]