SME Server:Documentation:QA:Verification

Not all bugs require code fixes but a substantial part of them do. An important part of the QA process is verifying the issue addressed in a bug is fixed when code is changed. We try our best not to overlook things but we might do, proper verification is important to reduce the risk of such things slipping through. Therefore we try and stick to a certain work flow when doing verification. Since the development team and resources are rather limited it is already hard to try and fix all bugs raised, let alone also verify them. Since you do not need to be a developer to verify the fixes we make this guide is written to help people from our community to help us in the process of verification.

General work flow

As already mentioned we try and stick to a certain work flow when verifying bugs. In general the process can be split in the following steps:

1. . Describe the environment briefly
2. . Show the installed versions of affected packages before upgrading
3. . Show you can reproduce the problem as reported in the bug
4. . Short synopsis of the resolution of the problem
5. . Update the affected packages
6. . Show the problem is fixed (and no ill side affects have appeared)
7. . Suggestion of the release note information
8. . Final conclusion: VERIFIED or REOPEN

Template

The above process is documented in the comment field of a bug using the following template, more information on the steps and the template sections can be found below.

= Environment:

= Original problem:

= Resolution:

= Current version installed:

= Testing 

= Updated version installed:

= Problem fixed:

= Verified or Reopen:

= Documentation Impact:

= Suggested Release Notes:

Environment

Since some bugs might appear in multiple versions of our products (e. g. SME Server 7.x as well as SME Server 8.x) we need to make sure you are doing verification using the proper environment. Most of the times it would be something like the following:

SME Server 7.5 fully updated, no contribs installed

Original problem

This section is used to show that you can reproduce the original problem on your test machine. It is important that you do this as accurate and methodological as possible as you will have to reproduce these steps after you have updated to the newer pacakage(s) to show that the problem is actually fixed. Make sure to make your test as extensive as required but on the other hand try and keep things as clear and dense as possible. In the case of New Feature Requests (NFR) or if a previous version of the package with the reported bug is not available this step might be skipped.

Resolution

This section is used to shortly describe which steps need to be taken in order to upgrade to the fixed version, usually it will be something like this:

Install package1 and package2 from smeupdates-testing and/or smetest

After that it is time to actually upgrade to the new package(s).

Current version installed

When the bug is reported and diagnosed normally the package that should be fixed, as well as the version of the package might be documented, but certainly the package name and the version on which the issue was fixed is reported most of the time, it will be something like below:

Fixed in package-name
* Thu Nov 25 2010 John Doe <jdoe@fqdn.org> x.y.z-r.sme
- Short description of the fix [SME: bugnumber]

The version intended here is the version of the package with the bug present, usualy the output of the rpm -q command will do, so most of the time this will look something like this:

[root@smetest ~]# rpm -q e-smith-base
e-smith-base-5.2.0-28.el5.sme
[root@smetest ~]#

Testing

Show in detail existing problem being fixed by new package

Updated version installed

This section is analogue to the Current version installed and is meant to show that the updated package(s) are in fact installed before verifying the bug has been fixed, usually a rpm -q for the updated package(s) would do.

Problem fixed

This is the heart of the verification process as here is where we show the problem is actually fixed. You can use the same steps for this as used in the Original problem section. Be sure to check that not only the problem is fixed, but also make sure no error messages are found in the logfiles or on the console when entering the comments. If errors are present please report them to the bug report if it affects the fix, or open a new bug when a new issue is discovered. Also make sure to test normal functionality related to the changes are still working properly.

Verified or Reopen

This is the section where we conclude on the verification process. If the fixed package works and there are no ill side effects we can conclude the package is VERIFIED, otherwise it might be REOPENED. If you have found other bugs in the process you can state them here (briefly) as well, but please keep in mind that new issues should be reported in a new bug report. When REOPENING the bug, please motivate this briefly.

	Note:
	Please, do not forget to also set the status field, at the bottom of the comment field, to the proper value matching your conclusion.

Documentation Impact

Eg: is a DB variable created/removed?

Suggested release notes

If you feel capable please suggest the information we can put in the release notes in one or two lines, if you cannot please leave this empty, or for instance specify it is to be determined by specifying T. B. D..