Difference between revisions of "SME Server:Documentation:Technical Manual:Booklet"
m (re link) |
m (Updated page name for Adding software) |
||
| (4 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| + | {{Ready_for_deletion}} | ||
{{ :FindingAnswers }} | {{ :FindingAnswers }} | ||
| − | {{ : | + | {{ :SME_Server:Volunteering }} |
{{ :DB_Variables_Configuration }} | {{ :DB_Variables_Configuration }} | ||
{{ SME_Server:Documentation:Technical_Manual:Chapter4 }} | {{ SME_Server:Documentation:Technical_Manual:Chapter4 }} | ||
{{ SME_Server:Documentation:Technical_Manual:Chapter5 }} | {{ SME_Server:Documentation:Technical_Manual:Chapter5 }} | ||
| − | {{ : | + | {{ :SME Software:Adding_Software }} |
| − | {{ : | + | {{ :Releasing Contribs }} |
| − | {{ : | + | {{ :Bugzilla Help }} |
Latest revision as of 00:07, 15 September 2012
Finding Answers
Ideally most of the information you need to run and configure your server will be in the manual If something you need isn't, work your way down the resources that follow. If it's appropriate, you can add the material to the wiki, if unsure ask.
Manuals
The Manuals for the User, Administrator and Developer.
Bug Tracker
Find, report and fix bugs here http://bugs.contribs.org
You don't have to be a programmer to help or use the bug tracker. Bugs also include fixing things and making suggestions on documentation, translations and the SME Server web sites.
Refer to Bugzilla Help for helping with bug fixing and verification.
Forums
There is an enormous amount of information here, a search can often find just the answer you are after. You can also find outdated, misinformed or wrong information.
The aim should be to transfer valuable information into the Manual or a Howto (with a link in the Howto List)
If you can't find the information you are after then post to the forum. If applicable, please document your answer in the wiki so it becomes a definitive answer and not just another hit on a search.
How To's
HowTo's are documents to solve a particular problem There is a list of HowTos here
Frequently Asked Questions
The Frequently Asked Questions page is to be used to document problems that cannot or will not be fixed through development of SME7. If a problem is fixed it should be moved into the Manual or a Howto.
Email Lists
There are a number of lists available to join at http://lists.contribs.org/mailman/listinfo
List descriptions and email archives are available here.
The Koozali SME Server project
The Koozali Foundation Inc. is a nonprofit corporation that governs the open source Koozali SME Server project. Koozali SME Server is a stable, secure and easy to use/manage linux server that provides common server functionalities out of the box. Many open source contributions are available that can extend the default server functionality making Koozali SME Server an even more powerful and flexible business server solution. Thousands of Koozali SME Severs have been deployed as real or virtual servers and in the cloud to serve many small to medium enterprises, and this number is growing day by day. The Koozali SME Server is free to use but it takes a lot of effort and money to develop, make, and maintain. We therefore ask you for your considerations.
Volunteering
Koozali Foundation Inc. together with its community hosted at https://contribs.org is a collaborative effort of volunteers. You too can contribute to the development and continuity of the Koozali SME Server project as described on our volunteering page. Everybody is welcome to join the already 4000+ member contribs.org community and can contribute with any skill set.
Financial donations
You can also show your support by making financial donations. The preferred way to make financial donations is using the donate option in the forums. You are free to choose any amount and frequency, being monthly, yearly or only once. The benefit of donating through your forums account is that your forum user name will receive a badge, showing your donation status. If you do not have a forum account, you can create one, or select the below PayPal option to make your donations.
Commercial usage
Organizations that use Koozali SME Server for their business, provide professional services related to SME Server or in any other way benefit commercially from the Koozali SME Server project, are kindly requested to consider regular financial donations that reflect their business benefits.
Koozali Foundation Inc. is happy to supply an invoice for any donations received. For more information on invoicing please send a mail to treasurer@koozali.org.
Thank you for your considerations and support!
Skills
Any skills are useful for the SME Server project.
If you are a developer of any kind, we will find something for you to do. Even if you think that you have no Linux skills you can still help. You can tell us if and how you understand the wiki pages, or more precisely, we would like to know what pages you don't understand. Thanks to the insights of new users we can continuously enhance readability of the wiki. Please add remarks to the discussion tabs of the page you are concerned with.
If, on top of that, you can write, you are invaluable. We need a lot of articles. There are thousands of Linux programs around, and more are released every day. In addition, new versions of existing applications are released. There is a need for a lot of hands to write about them and to keep reviews current.
Documentation
Documentation for SME Server was inherited from the prior distribution maintainers, e-smith and mitel. Their work gave SME a great base to work from.
The current developers have continued to improve the SME Server software and to reflect these improvements the Documentation has to develop too. These wiki-based manuals have been put in place to allow anyone to update or add new sections where they see fit.
The core manuals have been protected for stability, but anyone can request access to add and update howto and contrib pages to the wiki. If you've got instructions, solutions to common problems, neat tips and tricks, or just a good way to explain something, we'd love to hear from you. You may want to join our Document mailing lists to discuss your plans and coordinate with other Docteam members.
Wiki Team
This page is built with MediaWiki. The concept of a Wiki is that anybody may add and modify content. On Help:Contents, find a quick tutorial if you're unfamiliar with wikis.
As long as you edit anything on the wiki you are de facto member of the Wiki Team.
To make edits, you need to log in to the wiki. In order to do this, you will need to open an account. The information that SME Server requires for opening an account is very limited and easy to do and can be found here.
After you log in, you will see a toolbar at the top of the page that is always available. There is a link in the toolbar that is labeled with your username. This takes you to your "personal page". Use this page to introduce yourself and don't be shy. Write a word or two about your interests, especially those related to SME Server and Linux. If nothing else, tell us how you learned about SME Server.
Bug and New Feature Verification
You can help improve SME Server by finding and reporting bugs. This helps us to make our software as reliable as possible.
Our bug tracking system, Bugzilla, is used for all SME Server Linux products. If you have never written a bug report, please refer to Bugzilla Help to learn what kinds of information make the report most useful. Your role in that is to report possible bugs only via the bug tracker, and to encourage others to do the same. Refer again to SME_Server:Documentation:QA:Verification for helping with bug fixing and verification, the best way to learn is to fix other peoples' problems.
Programming
The most obvious way, for programmers, to participate in the development of SME Server is to post a patch as a suggested solution to an existing bug in Bugzilla. Each package has a maintainer, who will contact you to discuss your proposed solution. You may want to join our development mailing lists before you start coding in order to discuss your plans and coordinate with other developers. WIP: Coming on-board
For more information about getting source code and building your own packages, read the SME Server:Documentation:Developers Manual.
Mailing Lists
The communication among different teams go through a software called Mailman for which you choose the relevant list that interest you. All messages will be received in your email box, you will only respond to the email address of the mailling (ie devinfo@contribs.org for example) that your mail is sent to all registered users on the list . you can choose your Mailing Lists
Languages
A new system translation interface has been installed to facilitate translations of SME Server core packages as well as SME Server contribs, for more information have a look at the Translations page. The translation framework is located at http://translate.contribs.org. You may want to join our Translation mailing lists before you start your work and coordinate with other people.
Contribute
There is an active community in the forum that help all levels of SME Server users. Please have a look at the forum for an overview at http://forums.contribs.org
Suggestions
Visit the New Feature Request (NFR) page in the bug tracker and add your suggestion.
Only software with an OSI-compliant open-source license will be added to the SME Server project.
Is this article helpful to you?
Please consider donating or volunteering
Thank you!
Database variables
SME Server comes with the most used parameters set as variables in its internal configuration databases. These variables are used to store values to be used in the final configuration files. Please, read the SME_Server:Documentation:Developers_Manual:Section2 to understand the template and database process.
These variables are useful to configure your system more easily, as you do not need to modify configuration files directly for most common cases. It also makes it possible to administer the server through its server-manager as the database variables are used to set and change configuration parameters. After editing, the configuration files must be regenerated and affected services need to be restarted.
For example, suppose you need to increase "memory-limit" in php.
You would simply execute these commands at the server console:
db configuration setprop php MemoryLimit 64M expand-template /etc/php.ini /etc/init.d/httpd-e-smith restart
The first line changes the value for the memory limit of PHP, the second line regenerates the configuration file and the last line will reload Apache (and subsequently also PHP as this is configured as a module of Apache).
The database system is based on a flat file system, but you should never edit them directly. Instead you should use the db command. More details on using the database system can be found in the SME Server Developer's Guide.
Setting db variables to default values
Any db variable that has a default value can be reset to the default by deleting the variable entirely, then re-initializing the default database values as follows:
config delprop <key> <prop> /etc/e-smith/events/actions/initialize-default-databases
Delete a property value
To delete the property
db accounts delprop <key> <prop>
Reset a property value
To reset to an empty value
db accounts setprop <key> <prop> ''
Concept of the signal-event command
Due to the efforts of the developers, you can further simplify the commands using the signal-event proccess.
For full details see SME_Server:Documentation:Developers_Manual:Section2
Overview of database variables
The next section describes the standard variables defined on SME Server. Please update this list with new standard variables in future SME Server versions.
The tables below have three columns. The first is the variable, the second is the target variable (located in the final configuration file), and the third is the default value.
A lot of the variables can be set using the server-manager but some can not. For example the variable DomainMaster for samba is not important here, because this can be set through server-manager. On the other hand, the variable RecycleBin is important, because it is not accessible through the server-manager.
Configuration files may use database values from a single configuration key, or may use multiple keys. The latter is the case for the /etc/rc.d/init.d/masq configuration file. This file takes it values from multiple database keys such as squid and masq.
It is also possible that multiple configuration files use the same key. An example of this is the httpd-admin key. This key has a variable TCPPort which is used in multiple files (/etc/httpd/admin-conf/httpd.conf and /etc/services).
AppleTalk (atalk)
Usage
db configuration setprop atalk variable value signal-event workgroup-update
| Variable | Target | Default |
|---|---|---|
| MaxClients | AFPD_MAX_CLIENTS | 20 |
Backup
Usage
db configuration setprop backup variable value signal-event conf-backup
| Variable | Target | Default |
|---|---|---|
| Device | $device | /dev/st0 |
| Eject | Logical operation | no |
Console Mode
Usage - Choose either login or auto DB variable.
config set ConsoleMode login signal-event post-upgrade signal-event reboot
| Variable | Target | Default |
|---|---|---|
| ConsoleMode | Console Setting | login |
Clam AntiVirus (clamav)
clamav
Usage
db configuration setprop clamav variable value signal-event clamav-update
| Variable | Target | Default |
|---|---|---|
| ArchiveBlockEncrypted | ArchiveBlockEncrypted | no |
| ArchiveBlockMax | ArchiveBlockMax | no |
| ArchiveMaxCompressionRatio | ArchiveMaxCompressionRatio | 300 |
| ArchiveMaxFiles | ArchiveMaxFiles | 1500 |
| ArchiveMaxFileSize | ArchiveMaxFileSize | 15M |
| ArchiveMaxRecursion | ArchiveMaxRecursion | 8 |
| Debug | Debug | no |
| DetectBrokenExecutables | DetectBrokenExecutables | no |
| FilesystemScanExclude | FilesystemScanExclude | /proc,/sys,/usr/share,/var |
| IdleTimeout | IdleTimeout | 60 |
| LeaveTemporaryFiles | LeaveTemporaryFiles | no |
| LogClean | LogClean | yes |
| LogTime | LogTime | yes |
| LogVerbose | LogVerbose | yes |
| MaxConnectionQueueLength | MaxConnectionQueueLength | 30 |
| MaxDirectoryRecursion | MaxDirectoryRecursion | 20 |
| MaxThreads | MaxThreads | 20 |
| ReadTimeout | ReadTimeout | 300 |
| ScanArchive | ScanArchive | yes |
| ScanHTML | ScanHTML | yes |
| ScanMail | ScanMail | yes |
| ScanOLE2 | ScanOLE2 | yes |
| ScanPE | ScanPE | yes |
| SelfCheck | SelfCheck | 1800 |
| StreamMaxLength | StreamMaxLength | 25M |
| Variable | Target | Default |
|---|---|---|
| Checks | Checks | 24 |
| DatabaseMirror | DatabaseMirror | db.local.clamav.net |
| DNSDatabaseInfo | DNSDatabaseInfo | current.cvd.clamav.net |
| LogVerbose | LogVerbose | yes |
| MaxAttempts | MaxAttempts | 6 |
clamd
Usage
db configuration setprop clamd variable value signal-event clamav-update
| Variable | Target | Default |
|---|---|---|
| MemLimit | MEMLIMIT | 1400000000 |
DHCP daemon (dhcpd)
Usage
db configuration setprop dhcpd variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| Bootp | bootp | deny |
| startDynamicIPRange | range | |
| endDynamicIPRange | range |
Note: the end of the dynamic IP range will be set to the value of 'endDynamicIPRange' minus the value of pptpd:sessions.
DNS Cache Forwarder (dnscache / dnscache.forwarder)
Usage
db configuration setprop dnscache variable value signal-event dns-update
or for some settings
signal-event console-save
| Variable | Target | Default | Options |
|---|---|---|---|
| CacheSize | CACHESIZE | 1000000 (SME9 10000000) | Variable |
| DataLimit | DATALIMIT | 3000000 (SME9 12000000) | Variable |
| Forwarder | Forwarder | not configured | a.b.c.d - address of remote DNS server |
| Forwarder | Forwarder2 | not configured | a.b.c.d - address of remote DNS server |
TinyDNS
Usage
db configuration setprop tinydns variable value signal-event dns-update
| Variable | Target | Default |
|---|---|---|
| ListenIP | IP | 127.0.0.1 |
| DataLimit | DATALIMIT | 300000 |
FlexBackup
Usage
db configuration setprop flexbackup variable value signal-event conf-backup
| Variable | Target | Default |
|---|---|---|
| Blocksize | $blksize | 32 |
| TapeBlocksize | $mt_blksize | 0 |
| BufferProg | $buffer | buffer |
| BufferMegs | $buffer_megs | 20 |
| erase_rewind_only | $erase_rewind_only | false |
| Type | $type | tar |
Horde (webmail)
Usage
db configuration setprop horde variable value
expand-template /home/httpd/html/horde/conf.menu.apps.php
| Variable | Target | Default |
|---|---|---|
| MenuArray | MenuArray | enabled |
expand-template /home/httpd/html/horde/config/conf.php
| Variable | Target | Default |
|---|---|---|
| Administration | Administration | disabled |
expand-template /etc/e-smith/templates/home/httpd/html/horde/config/prefs.php/200personal
| Variable | Target | Default |
|---|---|---|
| Name | 'My Company' | 'Horde Webmail' |
expand-template /home/httpd/html/horde/turba/config/sources.php
| Variable | Target | Default |
|---|---|---|
| freebusy | freebusy | disabled |
| SharedAddressBooks | SharedAddressBooks | disabled |
Apache server ibay specific (httpd-e-smith)
see PHP for specific php options for ibays, or see Webhosting contrib.
Usage
db accounts setprop ibayname variable value signal-event ibay-modify ibayname
| Variable | Target | Default |
|---|---|---|
| AllowOverride | AllowOverride | None |
| FollowSymLinks | FollowSymLinks | disabled |
| Indexes | Indexes | enabled |
| PHPRegisterGlobals | register_globals | disabled |
| PHPBaseDir | open_basedir | /home/e-smith/files/ibays/ibayname |
| SSLv2 | SSLProtocol | disabled |
| SSL | Force https access to ibay through Apache. | disabled |
- these options are specific to SME Server 9 and are not backported to SME Server 8. See
bugzilla:8239
Usage
db accounts setprop ibayname variable value signal-event ibay-modify ibayname
Apache server-manager (httpd-admin)
| Variable | Target | Default |
|---|---|---|
| PermitPlainTextAccess | no | |
| ValidFrom | ip/mask coma separated list |
Usage
db configuration setprop httpd-admin variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| TCPPort | TCPPort | 980 |
IMAP (imap)
Usage
db configuration setprop imap variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| ConcurrencyLimit | INSTANCES | 2000 |
| ConcurrencyLimitPerIP | INSTANCES_PER_IP | 12 |
| ProcessMemoryLimit | ulimitdata | 128000000 |
- only for SME Server 9
| Variable | Target | Default |
|---|---|---|
| AllowPlainText | if set to disabled, dovecot will still listen on port 143, but will only accept TLS connexions, even from the local networks | enabled/disabled, default is enabled |
IMAPS (imaps)
These properties apply to SME versions before 9.0 only. After 9.0, the imap properties are used to control imaps concurrency and memory limits.
Usage
db configuration setprop imaps variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| ConcurrencyLimit | INSTANCES | 2000 |
| ConcurrencyLimitPerIP | INSTANCES_PER_IP | 12 |
| ProcessMemoryLimit | ulimitdata | 128000000 |
Dovecot
- Only for SME Server 9
With smeserver-dovecot installed, 4 services in the configuration DB are used
imap and imaps are used to be backward compatible with e-smith-imap (and are used to control the TCPPort of the service, and if it's accessible from local network or from the internet)
dovecot is now the main service entry in the configuration DB. It's used to control various optional features of dovecot
Usage
db configuration setprop dovecot variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| AdminIsMaster | if enabled, the admin user will be a master user, and will be able to login as any user. To do so use user1*admin as login and the admin password to log as user1 | enabled/disabled, default is disabled |
| FullTextIndexing | will turn on or off the full text indexing. When this option is enabled, a first search in an IMAP folder will trigger indexation. Next searches will be much faster. Read this page before enabling this option | enabled/disabled, default is disabled |
| LogActions | will turn on or off extra logging (flag change, move, copy etc…). !! Warning !!: enabling this can generate a huge amount of logs | enabled/disabled, default is disabled |
| Quotas | will report the actual used space and the remaining one if the user has a quota limit | enabled/disabled, default is enabled |
Fetchmail
Various fetchmail settings for email collection
Usage
db configuration setprop fetchmail variable value signal-event email-update
See the man page for more settings:
https://www.fetchmail.info/fetchmail-man.html
| Variable | Target | Default |
|---|---|---|
| Verbosity | For debugging | silent/verbose, default is silent |
| SSL | Use SSL | enabled/disabled, default is disabled |
| Protocol | POP3 | POP/Other, default is POP3 |
| TCPPort | Retrieved from smtpd | default 25 |
IPTables firewall (masq)
Usage
db configuration setprop masq variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| Logging | Logging | most |
| Stealth | Stealth | no |
Additional information on customizing iptables
Create a custom-named service definition in the configuration database.
db configuration set <servicename> service
Apply your desired firewall restrictions to any existing SME 'service' or to a custom-named service that you have created. Combine a custom-named service with port-forwarding to create customized firewall rules.
db configuration setprop <servicename> TCPPort <portnumber> db configuration setprop <servicename> TCPPorts <portnumbers> db configuration setprop <servicename> UDPPort <portnumber> db configuration setprop <servicename> UDPPorts <portnumbers> db configuration setprop <servicename> status enabled|disabled db configuration setprop <servicename> access public|private|localhost db configuration setprop <servicename> AllowHosts a.b.c.d,x.y.z.0/24 db configuration setprop <servicename> DenyHosts e.f.g.h,l.m.n.0/24
Effectuate the changes you have made
signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| TCPPort | --proto tcp --dport <Ports> | Pre-configured for default services; no default for custom services |
| TCPPorts | --proto tcp --dports <Ports> | No default for custom services; Ranges of ports are defined with a : not a - |
| UDPPort | --proto udp --dport <Ports> | Pre-configured for default services; no default for custom services |
| UDPPorts | --proto udp --dports <Ports> | No default for custom services; Ranges of ports are defined with a : not a - |
| status | disabled | AllowHosts is set to "" (an empty string) unless the status is 'enabled' |
| access | private | AllowHosts is set to "" (an empty string) unless access is 'public' |
| AllowHosts | --src ..... --jump ACCEPT | Pre-configured for default services; no default for custom services. Default is '0.0.0.0/0' if service is enabled and public. |
| DenyHosts | --src ..... --jump denylog | Pre-configured for default services; no default for custom services. If 'DenyHosts' is empty or does not exist then there are no '... --jump denylog' entries created in /etc/init.d/masq. |
SpamAssassin
Usage
db configuration setprop spamassassin variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| DNSAvailable | dns_available | yes |
| OkLanguages | ok_languages | all |
| OkLocales | ok_locales | all |
| ReportSafe | report_safe | 0 |
| Subject | rewrite_header Subject | [SPAM] |
| SkipRBLChecks | skip_rbl_checks | 0 |
| TrustedNetworks | trusted_networks | 127. |
| UseAutoWhitelist | use_auto_whitelist | 0 |
| UseBayes | use_bayes | 0 |
| Sensitivity | required_hits | medium |
Sometimes certain spamassassin update servers get corrupted or are not updated frequently. The list is available at: /var/lib/spamassassin/3.003001/updates_spamassassin_org/MIRRORED.BY
MySQL (mysqld)
Usage
db configuration setprop mysqld variable value expand-template /etc/my.cnf sv t /service/mysqld
| Variable | Target | Default |
|---|---|---|
| InnoDB | InnoDB | disabled |
| LocalNetworkingOnly | LocalNetworkingOnly | yes |
Network Time Protocol (ntpd)
Usage
db configuration setprop ntpd variable value signal-event timeserver-update
| Variable | Target | Default |
|---|---|---|
| MemLimit | MEMLIMIT | 35000000 |
| Variable | Target | Default |
|---|---|---|
| NTPServer | server | pool.ntp.org |
| SyncToHWClockSupported | SyncToHWClockSupported | yes |
SupportLargeDrift
A new db key for ntpd: SupportLargeDrift.
Default value is disabled, which doesn't change the current behaviour. bugzilla: 7979
If set to enabled, it will - add tinker panic 0 at the begening of the ntp.conf - remove the lines
server 127.127.1.0 # local clock fudge 127.127.1.0 stratum 10
With SupportLargeDrift enabled, the guest is able to resync the clock with the configured ntp server, even after resuming from a suspended state (tested with a ~10min drift, it took about 3 or 4 minutes for the guest to resync the clock after resuming).
db configuration setprop ntpd SupportLargeDrift enabled
Php
see PHP page for all the available options
Usage
db configuration setprop php variable value expand-template /etc/php.ini /etc/init.d/httpd-e-smith restart
| Variable | Target | Default |
|---|---|---|
| MaxExecutionTime | max_execution_time | 30 |
| MemoryLimit | memory_limit | 32M |
| PostMaxSize | post_max_size | 20M |
| UploadMaxFilesize | upload_max_filesize | 10M |
| AllowUrlFopen | allow_url_fopen | Off |
| ExposePHP | expose_php : Exposes to the world that PHP is installed on the server | Off |
Don't forget "M" unit because you get a lot of httpd errors and apache can't start!
| Variable | Target | Default |
|---|---|---|
| AllowUrlFopen | AllowUrlfOpen | disabled, set to enabled |
| MemoryLimit | MemoryLimit | disabled, set a M as unit, eg 64M |
| UpMaxFileSize | UpMaxFileSize | disabled, set a M as unit, eg 64M |
| PostMaxSize | PostMaxSize | disabled, set a M as unit, eg 64M |
| MaxExecTime | MaxExecTime | disabled, set time in second without units, eg 60 or unlimited |
Virtual Private Network (VPN) (pptpd)
Usage
db configuration setprop pptpd variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| debug | debug | no |
| mtu | mtu | not set by default, add your value (1404) after mtu |
| mru | mru | not set by default, add your value (1404) after mru
- |
| Passive | passive | enabled |
| Interfaces | Unknown | not set by default |
| Variable | Target | Default |
|---|---|---|
| debug | debug | no |
Pro FTP (proftpd)
Usage
db configuration setprop ftp variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| DisableAnonymous | DisableAnonymous | no |
Qmail
You can set the maximum size of email to be sent
Usage expressed in bytes
db configuration setprop qmail MaxMessageSize 15000000 signal-event email-update
| Variable | Target | Default |
|---|---|---|
| MaxMessageSize | The maximum email size for sending | 15000000 |
Qpsmptd
Work in progress !!
Usage
config show qpsmtpd
config setprop qpsmtpd variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| Authentication | Authentication | enabled |
| Bcc | Bcc | disabled |
| BccMode | BccMode | cc |
| BccUser | BccUser | maillog |
| DKIMSigning | DKIMSigning | enabled |
| DNSBL | DNSBL | disabled |
| Instances | Instances | 40 |
| InstancesPerIP | InstancesPerIP | 5 |
| LogLevel | LogLevel | 6 |
| MaxScannerSize | MaxScannerSize | 25000000 |
| MaximumDateOffset | MaximumDateOffset | 0 |
| PatternScan | PatternScan | disabled |
| Proxy | Proxy | blocked |
| RBLList | RBLList | bl.spamcop.net,dnsbl-1.uceprotect.net,dnsbl-2.uceprotect.net,psbl.surriel.com,zen.spamhaus.org |
| RHSBL | RHSBL | disabled |
| RelayRequiresAuth | RelayRequiresAuth | enabled |
| SBLList | SBLList | multi.surbl.org,black.uribl.com,rhsbl.sorbs.net |
| TCPPort | TCPPort | 25 |
| TCPProxyPort | TCPProxyPort | 25 |
| TlsBeforeAuth | TlsBeforeAuth | 1 |
| UBLList | UBLList | multi.surbl.org:8-16-64-128,black.uribl.com,rhsbl.sorbs.net |
| URIBL | URIBL | disabled |
| VirusScan | VirusScan | enabled |
| access | access | public |
| qplogsumm | qplogsumm | disabled |
| status | status | enabled |
| tnef2mime | tnef2mime | enabled |
Samba global settings (smbd)
Usage
db configuration setprop smb variable value signal-event ibay-modify
| Variable | Target | Default |
|---|---|---|
| RecycleBin | recycle | disabled |
| ShadowCopy | shadow_copy | disabled |
| DeadTime | deadtime | 10080 |
| DisplayCharSet | display charset | ISO8859-1 |
| DosCharSet | dos charset | 850 |
| LogonDrive | logon drive | Z |
| OpLocks | oplocks | enabled |
| OsLevel | os level | 65 |
| ServerString | server string | SME Server |
| SMBPorts | smb ports | 139 |
| UnixCharSet | unix charset | UTF8 |
| UseClientDriver | use client driver | yes |
| LogLevel | log level | 1 |
Samba per i-bay settings (smbd)
Usage
db accounts setprop ibay_name variable value signal-event ibay-modify
| Variable | Target | Default |
|---|---|---|
| Browseable | browseable | enabled |
| OpLocks | oplocks | enabled |
| RecycleBin | recycle | disabled |
| VetoOplockFiles | veto oplock files | (not set) |
| Audit | full_audit | disabled |
| KeepVersions | If RecycleBin is enabled in smbd, then you can keep version of recycle bin | disabled, set it to enabled |
| ShadowCopy | If Shadowcopy is enabled in the smbd, then you can turn off per ibay | enabled, set it to disabled |
| cscPolicy | set the csc policy (manual, documents, programs, disable) | (not set) |
Squid Proxy (squid)
Usage
db configuration setprop squid variable value signal-event proxy-update
| Variable | Target | Default |
|---|---|---|
| SSLPorts | Configure additional https ports (use single port or multiple ports separated by coma (,) | no default value (443 and 563 are hard coded) |
| SafePorts | acl Safe_ports port | 80 |
| EnforceSafePorts | EnforceSafePorts | no |
How to configure additional https ports
- only one port
config setprop squid SSLPorts 2083 signal-event proxy-update
- several ports
config setprop squid SSLPorts 2083,569,1,568,965 signal-event proxy-update
- remove ports
config setprop squid SSLPorts "" signal-event proxy-update
| Variable | Target | Default |
|---|---|---|
| Transparent | Transparent | yes |
| Variable | Target | Default |
|---|---|---|
| TransparentPort | TransparentPort | 3128 |
Alternate Usage for Configuration of an Up-Stream Proxy Server
db configuration set squid-parent-variable value signal-event proxy-update
| squid-parent-variable | Target | Default |
|---|---|---|
| SquidParent | name-or-ip-of-upstream-proxy-server | (none) |
| SquidParentPort | port-number-used-by-upstream-proxy-server | (none) |
(un-do using 'db configuration delete SquidParent', 'signal-event proxy-update')
SSH (sshd)
Usage
db configuration setprop sshd variable value signal-event remoteaccess-update
| Variable | Target | Default |
|---|---|---|
| TCPPort | Port | 22 |
| Protocol | Protocol | 2 |
| UsePAM | UsePAM | no |
| MaxAuthTries | MaxAuthTries | 2 |
| MaxStartups | MaxStartups | 10:30:60 |
| MotdStatus | MotdStatus (display or not the motd) | enabled |
| PasswordAuthentication | PasswordAuthentication | no |
| PermitRootLogin | PermitRootLogin | no |
| AllowHosts | AllowHosts | IP address(es) list |
Autoblock_ssh
see AutoBlock#Public_SSH_Acess
| Variable | Target | Default |
|---|---|---|
| AutoBlockTime | AutoBlockTime | 900 |
| AutoBlockTries | AutoBlockTries | 4 |
| AutoBlock | AutoBlock | enabled for sme9/disabled for sme8 |
smtpd
Usage
config setprop smtpd variable value signal-event email-update
| Variable | Target | Default |
|---|---|---|
| Instances | Total smtp Instances | 40 |
| InstancesPerIP | smtp-Instances-Per-IP | 5 |
| Variable | Target | Default |
|---|---|---|
| Greeting | Hostname portion of the greeting provided by your server to inbound SMTP connections | $SystemName.$DomainName |
| Variable | Target | Default |
|---|---|---|
| HeloHost | SMTP Helo / Ehlo value provided by your server when connecting to external SMTP servers to send email | $DomainName |
yum
Usage
config setprop yum variable value signal-event yum-modify
| Variable | Target | Default |
|---|---|---|
| AutoInstallUpdates | Install updates automatically? | disabled |
| check4updates | Frequency of Update Checking daily(default but monthly or weekly available) | daily |
| EnableGroups | Enable Groups | 0 |
| GPGCheck | Check GPG signature for repositories | 0 |
| PackageFunctions | Display individual packages in 'Software Installer' | disabled |
| RandomDelay | Random Delay | 120 |
| status | Yum's status | enabled |
| RestrictRepo | Repo names whose contents should be excluded from 'Available Packages' in the 'Software Installer' | none |
| RestrictRPM | All or part of an RPM name to be excluded from 'Available Packages' in the 'Software Installer' | none |
| DeltaRpmProcess | Only changes between the installed package and the new one are downloaded. Once the delta rpm loaded, a rebuilding process is started only SME10 see |
disabled (by default)/enabled |
| DownloadOnlyHour XX (0-23) | Set the time when to download rpm updates by yum (only sme10 see [bugzilla:1502]]) | default is 04 AM if no property |
See also 'db yum_repositories' All available repositories
Usage
db yum_repositories setprop RepositoryName variable value signal-event yum-modify
| Variable | Target | Default |
|---|---|---|
| EnableGroups | Enable groupinstall with yum | Yes(default)/no |
| GPGCheck | Enable the rpm verification by GPG of the repository signature | Yes(default)/no |
| MirrorList | It is the base url where the repository can be found | no default value |
| status | Enable the repository in yum, all updates will be installed if enabled | disabled/enabled |
| Visible | The repository can be selected from 'Enabled repositories' in the 'Software Installer' in order to be Enabled by Yum if set to yes | no |
| IncludePkgs 'rpm1,rpm2,rpm3' | Only rpms mentioned here will be available for installation or upgrade. | |
| Exclude 'rpm1,rpm2,rpm3' | rpms mentioned here will be excluded by yum | |
| DeltaRpmPercentage XX | Defines the maximum ratio allowed between the delta rpm size and the package size on a per-repository basis: by default, delta rpms can’t be bigger than 75% of the size of the associated rpms, otherwise they are not used. Set to disabled if you don't want to use deltarpm for this repository (only SME10 see |
default is '75' if no property |
Miscellaneous Other DB Variables
| Command | service(s) | config file(s) | notes |
|---|---|---|---|
| db domains setprop test.com MailServer a.b.c.d or use FQDN in place of a.b.c.d eg db domains setprop test.com MailServer aspmx.l.google.com |
qpsmtpd; qmail; fetchmail | /var/service/qpsmtpd/config/goodrcptto
/var/service/qpsmtpd/config/peers/local /var/service/qpsmtpd/config/peers/ /var/service/qpsmtpd/plugins /var/service/qmail/control/virtualdomains /var/service/qmail/control/smtproutes /etc/fetchmail |
Forward all email for the specified domain to the IP address a.b.c.d. a.b.c.d can be either local or remote. By default, the recipient address will be verified as valid on a.b.c.d before SME accepts the inbound message. |
| config set SquidParent <hostname or IP> | squid, diald | /etc/diald.filter, /etc/squid/squid.conf | Configure squid to peform all web downloads from the specified upstream proxy server |
| config set SquidParentPort <portnumber> | squid | /etc/squid/squid.conf | Connect to the upstream proxy server using <portnumber>. Defaults to 3128 if 'SquidParentPort' is unspecified. Ignored if SquidParent is not set. |
| config delete SquidParent | squid, diald | /etc/squid/squid.conf, /etc/diald.filter | Return squid to normal operation (no upstream proxy server) |
| db accounts setprop username Visible internal ; signal-event email-update | n/a | n/a | Make an email address invisible from outside? (see http://forums.contribs.org/index.php?topic=36302.0) |
| db accounts setprop pseudonym Visible internal ; signal-event email-update | n/a | n/a | Make an pseudonym email address invisible from outside |
| db <database> delprop key property ; /etc/e-smith/events/actions/initialize-default-databases | various | various | Restore the developers' default value for property |
| db <database> delete key ; /etc/e-smith/events/actions/initialize-default-databases | various | various | Restore the developers' default value for each property belonging to the key key |
| config set AdminIsNotRoot enabled | n/a | n/a | In server-manager panel, changing admin password no more change root password. root password is managed through passwd shell command and admin and root passwords can be distinct passwords. |
| config setprop smtp-auth-proxy PeerPort xxx; signal-event email-update | smtp-auth-proxy | none - the smtp-auth-proxy executable (//usr/local/sbin/smtp-auth-proxy.pl) reads the config database directly. | Used to change the port number used to connect to the upstream mail server ("SMTPSmartHost" or "Address of Internet provider's mail server"). Defaults to port 25 if PeerPort is not set; uses SSL if port 465 is selected. |
| db configuration setprop qpsmtpd tlsCipher XXX; signal-event email-update | qpsmtpd | /var/service/qpsmtpd/config/tls_ciphers | By default qpsmtpd only accepts the stronger SSL 3.0 or TLS 1.0 protocols for securing SMTPS connections. If needed, one can set qpsmtpd to also allow the weaker SSL 2.0 protocol. For XXX one can use: 'ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM' (SSLv2/SSLv3/TLSv1) 'HIGH:!SSLv2' (=Default: only allow stronger SSLv3/TLSv1 protocols) Note: don't forget to use the quotes!! |
| config setprop pppoe Mlimit <value> | pppoe | /service/wan/run.pppoe.conf | notes. - <value> cannot be set below 100000000 - <value> can be set above 100000000.
If pppoe Mlimit is set to a value below the MIN_MEMORY_LIMIT, currently 100000000, this lower value will not be accepted and Mlimit will be set to the default value (100000000). |
| command | service(s) | config file(s) | notes. Copy this block when adding new entries to this table. |
Port Forwarding
Server manager will create two databases, one for TCP and one for UDP
db portforward_tcp set {port} forward AllowHosts {some.host.ip} Comment {Test} Denyhosts {0.0.0.0/0} DestHost {dest.host.ip} DestPort {port}
db portforward_udp set {port} forward AllowHosts {some.host.ip} Comment {Test} Denyhosts {0.0.0.0/0} DestHost {dest.host.ip} DestPort {port}
Apply with:
signal-event portforwarding-update
| Variable | Target | Default |
|---|---|---|
| port | Incoming Port for Forwarding | none |
| DestPort | Destination Target Port | port |
| DestHost | Destination Host IP | none |
| AllowHosts | Allowed Hosts | 0.0.0.0/0 |
| DenyHosts | Denied Hosts | 0.0.0.0/0 |
| Comment | Notes for this rule | none |
Chapter 4. Software Configuration
Information for software that comes with the SME Server but may require additional configuration
Is this article helpful to you?
Please consider donating or volunteering
Thank you!
Uninterruptable Power Supply
Introduction
The primary goal of the Network UPS Tools (NUT) project is to provide reliable monitoring of UPS hardware and ensure safe shutdowns of the systems which are connected.
The default configuration of NUT, will keep your connected systems operational until a critical battery state is reached (ie battery is nearing exhaustion) and then power down your server/equipment in a controlled fashion. See http://www.networkupstools.org/
If you have an APC UPS, see also Uninterruptable_Power_Supply:APC for an alternative to the standard SME Nut implementation
If you have Dell UPS, this might Help Uninterruptable_Power_Supply:LatestGeekery
Default Configuration (USB)
The default configuration in SME Server for 'NUT' is set by the configuration database properties
Model = usbhid-ups status = disabled type = service
Most USB connected UPS's will work with these default settings. If using a USB connection just enable NUT as follows:
For SME 10, systemd is now in use. We have created a generic service for your convenience:
config setprop nut status enabled signal-event e-smith-nutUPS-update systemctl restart nut-server.service systemctl restart nut-monitor.service systemctl restart nut.service
config setprop nut status enabled signal-event post-upgrade signal-event reboot
If your USB UPS does not work properly OR you have a serial device then follow the Configuration Options below as required.
Configuration Options
Not all UPS's are supported by USB or the usbhid-ups driver. However NUT supports many UPS's and can be configured under SME Server easily.
Serial Connection
- Find the configuration details for your model of UPS. Refer to: http://www.networkupstools.org/stable-hcl.html and make note of the driver name and upstype number (if any) in the third column.
- From the console issue the following commands:
config setprop nut Model <model>
config setprop nut Device <device>
config setprop nut Type <type>
config setprop nut status enabled
Where:
<model> and <type> are the driver name and type number found above.
<device> is the serial port that the UPS is connected to eg. /dev/ttyS0. It also possible to use a more readable symlink. See HowTo on udev - symlinks for details. Note: The case of Model, Device and Type. - Check: config show nut
- Apply changes and restart server: signal-event post-upgrade signal-event reboot Alternatively, without NUT running or requiring a server reboot: signal-event console-save service nut start
- Confirm server is communicating with UPS: upsc UPS@localhost Whenever a UPS event occurs Emails are sent to the admin account.
Configuring as a master
edit the file /etc/ups/nut.conf, and modify the line
MODE=netserver
Set configuration values:
config setprop nut access private TCPPort 3493 expand-template /etc/ups/upsd.conf expand-template /etc/rc.d/init.d/masq systemctl restart masq systemctl restart nut-server
Configuring as a slave
Set configuration values:
config setprop nut SlaveUPS UPS@192.168.33.11 config setprop nut Master no
Where 192.168.33.11 is your UPS master, that is the computer that is in direct communication with the UPS. The hostname of that computer may also work.
Apply changes and restart server:
signal-event post-upgrade signal-event reboot
or (for SME10):
signal-event e-smith-nutUPS-update
Confirm server is communicating with master:
upsc UPS@192.168.33.11
Connecting multiple UPS's
As reference to http://bugs.contribs.org/show_bug.cgi?id=629 and https://bugs.koozali.org/show_bug.cgi?id=626#c2
1- you will need to do
mkdir -p /etc/e-smith/templates-custom/ups/ups.conf/
cp /etc/e-smith/templates/ups/ups.conf/UPS /etc/e-smith/templates-custom/ups/ups.conf/UPS2
then edit content to replace the header to UPS2 and Model and Device to Model2 and Device2
2- Then you need to do
mkdir -p /etc/e-smith/templates-custom/ups/upsmon.conf/
cp /etc/e-smith/templates/ups/upsmon.conf/MONITOR /etc/e-smith/templates-custom/ups/ups.conf/MONITOR2
Then edit it to change UPS to UPS2. Then you set Device2 and Model2.
3- Repeat the steps 1 and 2 for as many UPS you have, then finish by
signal-event console-save
UPS Variables and Commands
In some cases you may wish to modify variables on the actual UPS such as the Low Charge/LOWBATTERY setting. This requires the use of the upsrw command and UPS administrative privileges.
You may also want to control the UPS directly from the command line by issuing UPS commands. This requires use of the upscmd command and UPS administrative privileges.
UPS Administrative Privileges
You should check your new password ( AdminPass ) to run upsrw & upscmd. Of course, you could change your password for a easier one to use.
config show nut
To set new admin password in database. The new password would be admin ( change it to suit your need )
config setprop nut AdminPass admin
To enabled administrative privileges and run command to ups.
config setprop nut AdminUser enabled
Now, to get upsd to recognise admin modification for administrative privileges we expand the template and reload the upsd configuration
/sbin/e-smith/expand-template /etc/ups/upsd.users /usr/sbin/upsd -c reload
rpm -qa e-smith-nutUPS
If you get NUT running with administrative privileges modification from sme8 instruction, you need to remove the custom template created for this. These variables are taken in charge by the new package.
First you need to delete the custom template file.( This command delete the directory, be sure you don't have personal file present. If the case, delete manualy the file created for administrative privileges )
rm -rf /etc/e-smith/templates-custom/etc/ups/upsd.users
Now, to get upsd to recognise the modification of user, we need to expand the template and reload the upsd configuration
/sbin/e-smith/expand-template /etc/ups/upsd.users /usr/sbin/upsd -c reload
You should check your new password ( AdminPass ) to run upsrw & upscmd. Of course, you could change your password for a easier one to use.
config show nut
To set new admin password in database. The new password would be admin ( change it to suit your need )
config setprop nut AdminPass admin
To enabled administrative privileges and run command to ups.
config setprop nut AdminUser enabled
Now, to get upsd to recognise admin modification for administrative privileges we expand the template and reload the upsd configuration
/sbin/e-smith/expand-template /etc/ups/upsd.users /usr/sbin/upsd -c reload
In order to be able to use upsrw and upscmd it is necessary to have a suitable additional user defined in the upsd.users configuration file.
In order to create a suitable user we will use SME Servers templating system and configuration database. This is based on an original forum thread[1].
First we need to create a suitable custom template directory
mkdir -p /etc/e-smith/templates-custom/etc/ups/upsd.users cd /etc/e-smith/templates-custom/etc/ups/upsd.users
Create and edit a new file called 'admin' with the following content:
{
# create admin user for upsd to allow setting of
# UPS parameters via upsrw
$OUT .= "";
return unless (($nut{AdminUser} || 'disabled') eq 'enabled');
return unless (($nut{AdminPass} || '') ne '');
$OUT .= "\n";
$OUT .= " [admin]\n";
$OUT .= " password = $nut{AdminPass}\n";
if ( ($nut{Master} || 'yes') ne 'no') {
$OUT .= " allowfrom = localhost\n";
} else {
$OUT .= " allowfrom = localhost localnet\n";
}
$OUT .= " actions = set\n";
$OUT .= " instcmds = all\n";
}
Create two new database properties for nut
config setprop nut AdminUser enabled (This enables the creation of the user in the template above) config setprop nut AdminPass admin (This sets a password for the admin user. Set to whatever you want)
Now, to get upsd to recognise the new user with the required administrative privileges we expand the template and reload the upsd configuration
/sbin/e-smith/expand-template /etc/ups/upsd.users /usr/sbin/upsd -c reload
UPS access
The access of the ups is controled by database properties. The default propertie is set to localhost and give permission to run upsrw & upscmd from localhost only if administrative privileges is set to enabled as above. No slave ups could be connected in this mode. Three choices is available to set access.
- localhost: the ups access is only from the local machine ( UPS master ).
- private: the ups access is from your local machine and local network as per define in server-manager panel.
- public: the ups access is similar to localhost.
To set access properties in the database ( example: localhost )
config setprop nut access localhost /sbin/e-smith/expand-template /etc/ups/upsd.conf /usr/sbin/upsd -c reload
In localhost or public mode ( no remote access ), access to your ups is ( UPS name is UPS )
UPS@localhost
In private mode, access to your ups is ( UPS name is UPS )
UPS@localhost or UPS@192.168.1.1 ( ups master IP ) slave ups get access with UPS@192.168.1.1 ( ups master IP )
Setting UPS Variables
In order to set UPS variables it is necessary to have enabled the administrative privileges as above first and you get the possibility to run command from slave ups if access is set to private as above.
In the examples below, it is assumed your UPS name is UPS, that it is local, that the administrative user is admin and password admin. You can verify your UPS name via:
upsc -l
To view a complete list of the UPS variables, both informational and modifiable
upsc UPS
To determine the modifiable variables for your UPS, their current settings and their available setting values execute the command:
upsrw UPS
You can now modify the variables you wish using a command similar to the following (Note the order of the arguments is important, and you may need quotes around the value being set, "20"):
upsrw -s battery.charge.low=20 -u admin -p admin UPS
For remote host (slave UPS ), we need to add the IP from master UPS to run command.
upsrw -s battery.charge.low=20 -u admin -p admin UPS@192.168.2.1
Where the value after -s should be one of the parameters identified by the upsrw ups command. You can of course verify your changes using
upsrw UPS
or
upsc UPS
After you are done, clean up by disabling the upsc administrative privileges:
More information on upsrw can be found at:
- Manual page: man upsrw
Changing battery date
An example to update you battery date upon changing it. (use your own password)
upsrw -s battery.mfr.date=2020/08/31 -u admin -p admin UPS
Issuing UPS Commands
In order to issue UPS commands it is necessary to have enabled the administrative privileges as above first and you get the possibility to run command from slave ups if access is set to private as above.
In the examples below, it is assumed your UPS name is UPS, that it is local, that the administrative user is admin and password admin. You can verify your UPS name via:
upsc -l
To view a complete list of available commands for your UPS:
upscmd -l UPS
You can now issue a command to the localhost UPS with similar to the following:
upscmd -u admin -p admin UPS test.battery.start
For remote host (slave UPS ), we need to add the IP from master UPS to run command.
upscmd -u admin -p admin UPS@192.168.2.1 test.battery.start
Where the command test.battery.start is a valid command for your UPS as previously determined by upscmd -l UPS. Depending upon the command issued you may get broadcast messages and emails relating to and confirming what the UPS is doing.
After you are done, clean up by disabling the upsc administrative privileges.
Scheduling Events
Shutdown Time Delay Example
By default NUT will issue a shutdown command as soon as it receives a low battery event from the UPS. There may be instances and installation configurations that require a shutdown sooner, or other events with timed or schedules outcomes. See the man pages etc for further info and example situations.
In essence the upsmon program monitors the relevant UPS and for each NOTIFYFLAG event in upsmon.conf takes immediate action as defined. In order to delay or schedule any actions, the events need to be passed to upssched which can set timers and schedule events.
The following changes to standard SME Server NUT configuration will shut down the server a specified time after receiving the "on battery" signal (the example given is for 2 minutes). It assumes you already have an enabled and working NUT configuration and UPS
To create a timed shutdown before the BATTLOW signal is received, it is necessary to configure upssched and have a script handle the UPS events (upsmon cannot do this).
First we need to create a new custom template directory:
mkdir -p /etc/e-smith/templates-custom/etc/ups/upsmon.conf cd /etc/e-smith/templates-custom/etc/ups/upsmon.conf
Create and edit a new file called 'NOTIFYCMD' with the following content:
NOTIFYCMD /usr/sbin/upssched
Expand the template:
/sbin/e-smith/expand-template /etc/ups/upsmon.conf
Now create another a custom template directory
mkdir -p /etc/e-smith/templates-custom/etc/ups/upssched.conf cd /etc/e-smith/templates-custom/etc/ups/upssched.conf
Create and edit a new file called '01CONFIG' with the following content:
CMDSCRIPT /sbin/e-smith/nutUPS.cmd PIPEFN /tmp/upspipe LOCKFN /tmp/upslock AT COMMBAD * EXECUTE commbad AT COMMOK * EXECUTE commok AT NOCOMM * EXECUTE nocomm AT ONBATT * EXECUTE powerout AT ONBATT * START-TIMER shutdownnow 120 AT LOWBATT * EXECUTE shutdowncritical AT ONLINE * CANCEL-TIMER shutdownnow AT ONLINE * EXECUTE powerup
In the above set the line AT ONBATT * START-TIMER shutdownnow 120 to how many seconds after ONBATT signal you want to shut down
Expand the template:
/sbin/e-smith/expand-template /etc/ups/upssched.conf
Create and edit a new script file at:
/sbin/e-smith/nutUPS.cmd
Add the following content:
#! /bin/sh
case $1 in
commbad)
/bin/echo "UPS communications failure on `date`." | /bin/mail -s"UPS communications LOST" admin
/usr/bin/wall "UPS communications failure."
;;
commok)
/bin/echo "UPS communications restored on `date`." | /bin/mail -s"UPS communications restored" admin
/usr/bin/wall "UPS communications restored."
;;
nocomm)
/bin/echo "UPS communications cannot be established on `date`." | /bin/mail -s"UPS uncontactable" admin
/usr/bin/wall "UPS communications cannot be established."
;;
powerout)
/bin/echo "Power failure on `date`." | /bin/mail -s"UPS on battery" admin
/usr/bin/wall "UPS on battery. Shutdown in 60 seconds...."
;;
shutdownnow)
/bin/echo "UPS has been on battery for 60 seconds. Starting orderly shutdown on `date`." | /bin/mail -s"UPS on battery for 60 seconds" admin
/usr/bin/wall "UPS has been on battery for 60 seconds. Shutting down NOW!!!!"
/usr/bin/sudo /sbin/e-smith/signal-event halt
;;
shutdowncritical)
/bin/echo "UPS battery level CRITICAL. Starting EMERGENCY shutdown on `date`." | /bin/mail -s"UPS battery CRITICAL" admin
/usr/bin/wall "UPS battery level CRITICAL. Shutting down NOW!!!!"
/usr/bin/sudo /sbin/e-smith/signal-event halt
;;
powerup)
/bin/echo "Power restored on `date`." | /bin/mail -s"UPS on line" admin
/usr/bin/wall "UPS on line. Shutdown aborted."
;;
*)
/bin/echo "Unrecognized command: $1"
;;
esac
Now make it executable by nut user
chmod 754 /sbin/e-smith/nutUPS.cmd chown root:nut /sbin/e-smith/nutUPS.cmd
Nut requires to use sudo for this process to work, so sudo needs configuring to enable the user nut. By default the /etc/sudoers file is not part of the SME Server template system. To workaround this create a custom template directory:
mkdir -p /etc/e-smith/templates-custom/etc/sudoers cd /etc/e-smith/templates-custom/etc/sudoers
To preserve the content of the original /etc/sudoers file copy that into the custom template directory:
cp /etc/sudoers 10sudoers
Create and edit a new file called '30nut' with the following content:
nut ALL=NOPASSWD: ALL
Then run:
/sbin/e-smith/expand-template /etc/sudoers
Finally to complete the process:
signal-event post-upgrade signal-event reboot
While testing with the SMEServer v9.1 (circa March 2016), the above nutUPS.cmd script with entries in 01CONFIG template fails due to lack of permissions at the shutdownnow case at:
/usr/bin/sudo /sbin/e-smith/signal-event halt
Tweaking the /etc/sudoers did not mitigate it.
The error in the /var/log/messages is:
Mar 14 13:22:16 svr01 upssched[3507]: Timer daemon started Mar 14 13:22:16 svr01 upssched[3507]: New timer: shutdownnow (120 seconds) Mar 14 13:25:16 svr01 upssched[3507]: Event: shutdownnow Mar 14 13:25:16 svr01 wall[3539]: wall: user nut broadcasted 1 lines (70 chars) Mar 14 13:25:16 svr01 upssched[3507]: exec_cmd(/sbin/e-smith/nutUPS.cmd shutdownnow) returned 1
However, the nut user has the necessary shutdown permissions:
/usr/bin/sudo -u nut /usr/bin/sudo /sbin/e-smith/signal-event halt
Configure Nut-cgi Monitor Scripts
The nut-cgi rpm contains scripts that can be run via the webserver to monitor the UPS(s).
Download and install
You have to enable the epel repository:
yum install smeserver-extrarepositories-epel -y
then install nut-cgi:
yum install --enablerepo=epel nut-cgi
then configure it the SME way: Edit file /etc/ups/hosts.conf and add.
mkdir -p /etc/e-smith/templates-custom/etc/ups/hosts.conf echo 'MONITOR UPS@localhost "local UPS"' >/etc/e-smith/templates-custom/etc/ups/hosts.conf/10localhost expand-template /etc/ups/hosts.conf
Httpd template
mkdir -p /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf cd /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf
Now edit and create a new file 92nutupscmon with the following content
{
$OUT = "";
my $allow = 'all';
my $pass = '0';
my $satisfy = 'all';
my $name = $nut{'Name'} || 'NUT UPS Daemon Monitoring';
my $PublicAccess = $nut{'PublicAccess'} || "local";
for ('exit-if-none')
{
if ($PublicAccess eq 'none')
{
next;
}
elsif ($PublicAccess eq 'local')
{
$allow = "ip $localAccess";
$pass = 0;
$satisfy = 'All';
}
elsif ($PublicAccess eq 'local-pw')
{
$allow = "ip $localAccess";
$pass = 1;
$satisfy = 'All';
}
elsif ($PublicAccess eq 'global')
{
$allow = 'all granted';
$pass = 0;
$satisfy = 'All';
}
elsif ($PublicAccess eq 'global-pw')
{
$allow = 'all granted';
$pass = 1;
$satisfy = 'All';
}
elsif ($PublicAccess eq 'global-pw-remote')
{
$allow = "ip $localAccess";
$pass = 1;
$satisfy = 'Any';
}
$OUT .= "#------------------------------------------------------------\n";
$OUT .= "# nut multimon - $name\n";
$OUT .= "#------------------------------------------------------------\n";
{
if ((exists $nut{'URL'}) && ($nut{'URL'} ne '')) {
$OUT .= "Alias /$nut{'URL'} /var/www/nut-cgi-bin\n";
}
}
$OUT .= "Alias /nut /var/www/nut-cgi-bin\n";
$OUT .= "\n";
$OUT .= "<Directory /var/www/nut-cgi-bin>\n";
$OUT .= " DirectoryIndex upsstats.cgi\n";
$OUT .= " Options +ExecCGI\n";
$OUT .= " <Require$satisfy>\n" if ($pass);
$OUT .= " Require $allow\n";
if ($pass)
{
$OUT .= " AuthName $name\n";
$OUT .= " AuthType Basic\n";
$OUT .= " AuthExternal pwauth\n";
$OUT .= " require valid-user\n";
$OUT .= " </Require$satisfy>\n";
}
$OUT .= "</Directory>\n";
}
}
Configure databases and expand the template
config setprop nut PublicAccess local /sbin/e-smith/expand-template /etc/httpd/conf/httpd.conf
Restart the web server
systemctl restart httpd-e-smith
You have to enable the epel repositories.
yum install --enablerepo=epel nut-cgi
Edit file /etc/ups/hosts.conf and add.
MONITOR UPS@localhost "local UPS"
The nut-cgi rpm contains three cgi scripts. The rpm does not install them correctly for SME however so the following modifications are needed.
mkdir -p /opt/nut-cgi-bin chown root:www /opt/nut-cgi-bin mv /var/www/nut-cgi-bin/upsstats.cgi /opt/nut-cgi-bin mv /var/www/nut-cgi-bin/upsset.cgi /opt/nut-cgi-bin mv /var/www/nut-cgi-bin/upsimage.cgi /opt/nut-cgi-bin chown root:www /opt/nut-cgi-bin/* chmod 750 /opt/nut-cgi-bin/* mkdir -p /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf cd /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf
Now edit and create a new file 92nutupscmon with the following content
{
$OUT = "";
my $allow = 'all granted';
my $pass = '0';
my $satisfy = 'All';
my $name = $nut{'Name'} || 'NUT UPS Daemon Monitoring';
for ('exit-if-none')
{
if ($nut{'PublicAccess'})
{
if ($nut{'PublicAccess'} eq 'none')
{
next;
}
elsif ($nut{'PublicAccess'} eq 'local')
{
$allow = $localAccess;
$pass = 0;
$satisfy = 'all';
}
elsif ($nut{'PublicAccess'} eq 'local-pw')
{
$allow = $localAccess;
$pass = 1;
$satisfy = 'all';
}
elsif ($nut{'PublicAccess'} eq 'global')
{
$allow = 'all';
$pass = 0;
$satisfy = 'all';
}
elsif ($nut{'PublicAccess'} eq 'global-pw')
{
$allow = 'all';
$pass = 1;
$satisfy = 'all';
}
elsif ($nut{'PublicAccess'} eq 'global-pw-remote')
{
$allow = $localAccess;
$pass = 1;
$satisfy = 'any';
}
}
$OUT .= "#------------------------------------------------------------\n";
$OUT .= "# nut multimon - $name\n";
$OUT .= "#------------------------------------------------------------\n";
{
if ((exists $nut{'URL'}) && ($nut{'URL'} ne '')) {
$OUT .= "Alias /$nut{'URL'} /opt/nut-cgi-bin\n";
}
}
$OUT .= "Alias /nut /opt/nut-cgi-bin\n";
$OUT .= "\n";
$OUT .= "<Directory /opt/nut-cgi-bin>\n";
$OUT .= " DirectoryIndex upsstats.cgi\n";
$OUT .= " Options +ExecCGI\n";
$OUT .= " order deny,allow\n";
$OUT .= " deny from all\n";
$OUT .= " allow from $allow\n";
if ($pass)
{
$OUT .= " AuthName $name\n";
$OUT .= " AuthType Basic\n";
$OUT .= " AuthExternal pwauth\n";
$OUT .= " require valid-user\n";
$OUT .= " Satisfy $satisfy\n";
}
$OUT .= "</Directory>\n";
}
}
Configure databases and expand the template
config setprop nut PublicAccess local /sbin/e-smith/expand-template /etc/httpd/conf/httpd.conf
Restart the web server
sv t httpd-e-smith
Usage of Nut-cgi Scripts
Now go to http://yourdomain.tld/nut to see the statistics and information for the UPS at localhost.
By editing /etc/ups/hosts.conf and adding additional network UPS details, nut-cgi can be used to monitor more than one UPS. By the modification above, only the localhost is monitored.
Additional Information
There are template fragments in /etc/e-smith/templates/etc/ups that control the config files located in /etc/ups. The default settings should be OK for most situations. The /etc/nut.conf file must be manually edited like mode=standalone as the templates do not touch this file. In this case it would be:
sed -e 's/^MODE.*/MODE=standalone/' -i /etc/ups/nut.conf
By default, NUT is configured for a USB connected UPS in Master mode, but is disabled. When enabled, NUT will monitor the UPS and take various actions when certain notifications are received. This is controlled by the /etc/ups/upsmon.conf file which among other things lists the notifications and the actions to be taken for each. For example an On Battery event is captured by the NOTIFYFLAG ONBATT entry and the following SYSLOG+WALL+EXEC command string. This string tells upsmon to write the event to the System Log, broadcast a message to all users via Wall, and execute the command denoted by the NOTIFYCMD entry.
SME Server sets the NOTIFYCMD to /sbin/e-smith/nutUPS.notify, and this executable file simply sends an email to the SME admin user with a notification of the event.
Apart from the various events that the UPS and upsmon may notify via the NOTIFYFLAGS a Low Battery event will automatically and immediately cause upsmon to issue the SHUTDOWNCMD as defined in upsmon.conf (signal-event halt) and set a flag POWERDOWNFLAG so it knows on future restart that it is a UPS recovery.
For information on configuration parameters:
man ups.conf man upsd.conf man upsd.users man upsmon.conf man upssched.conf
For general information:
man upsd man nutupsdrv
Timeout Issues
If you have comms problems like this you can add a custom timeout:
"USBDEVFS_CONTROL failed cmd blazer_usb rqt 33 rq 9 len 8 ret -110"
Add a new config item. The default is 2
config setprop nut pollInterval 4
Modify the template
mkdir -p /etc/e-smith/templates-custom/etc/ups/ups.conf cp /etc/e-smith/templates/etc/ups/ups.conf/UPS /etc/e-smith/templates-custom/etc/ups/ups.conf/UPS nano /etc/e-smith/templates-custom/etc/ups/ups.conf/UPS
Add the bits between the # comments
{
my $model = $nut{Model} || "usbhid-ups";
my $device = $nut{Device} || "/var/lib/ups/hiddev0";
my $type = $nut{Type};
my $mfr = $nut{mfr};
my $mdl = $nut{mdl};
# Add this
my $poll = $nut{pollInterval} || '2';
if ($poll ne '2') {
$OUT .= "pollinterval = $poll\n";
}
# ends here
$OUT .= "[UPS]\n"; $OUT .= "\tdriver = $model\n";
expand-template /etc/ups/ups.conf cat /etc/ups/ups.conf
You should see something like this:
# Copyright (C) 1999-2006 Mitel Networks Corporation #------------------------------------------------------------ pollinterval = 4 [UPS]
Restart nut
service nut restart
Now check to see the correct timeout:
upsc UPS | grep driver.parameter.pollinterval
driver.parameter.pollinterval: 4
To reset either delete the key, or set it to the default of 2
Further reading
The NUT website is here: NUT
From the above references you can glean which configuration setting does what function, etc.
If you want to modify the operation of NUT from the standard configuration, then you should generally modify the NUT config files by creating custom templates, expanding the templates and restarting service. This will ensure modifications survive a future reboot or reconfiguration.
An example of doing this can be found in the forum [2] and in the section above for UPS Administrative Privileges
See also Known Problem - Restarting Nut
Documentation
Nut is a Software well documented, you can find the TOC here and with an overview
SME Server up to and including version 9.x runs MySQL as a database server.
SME Server 10 uses MariaDB to provide this function. A lot of applications require a MySQL database, among them is the Horde webmail interface which is supplied by SME Server by default.
- MySQL website: http://www.mysql.com
- MySQL 4.1 manual: http://dev.mysql.com/doc/refman/4.1/en/
- MariaDB documentation: https://mariadb.org/documentation/
General
The SME Server is based on CentOS, the development team will take their stock RPM's from the CentOS releases. The current version of MariaDB installed on SME Server is version 5.5.68.You can upgrade MariaDB, using their rpms, to a higher version but you are advised not to do so, as this might break your SME Server configuration. The Horde webmail interface relies on MariaDB. Upgrading to version 10.x has potential to break stuff like webmail. If you insist on upgrading MariaDB you may be able to find instructions in the forum, but be advised that no support can be expected from the developers and all bugs reported in the bugtracker will not be taken into account.
Alternatively you can rely on contribs and Red-Hat Software collection to add MySQL 5.7 and MariaDB 10.1 10.2 10.3 or 10.5 as secondary SQL service to satisfy your needs.
MariaDB on SME Server runs on a socket instead of on a port which you might be accustomed to. This is done to improve security as in the view of the development team only the server itself (localhost) needs to have access to the MySQL server. However you can configure MySQL to be accessible from the local network (see below).
[mysqld] [mysqld_safe] [mysql-5.7] [mariadb-10.1] [mariadb-10.2] [mariadb-10.3] [mariadb-10.5]
Access to MariaDB/MySQL from my application
As stated above on SME Server you have to use socket, this is more secure than using port. By default the service only listen on the server using socket, so trying to connect with any port will result in a failure.
Most application will have to define a string to access the socket, as below pointing to localhost (not 127.0.0.1, nor the LAN ip) and the full path to the socket. In some situation you will have to define the socket path and the host (localhost again and not 127.0.0.1) in variables.
define( 'DB_HOST', 'localhost:/var/lib/mysql/mysql.sock' );
MariaDB/MySQL root password
There appears to be no password set for the MySQL root password, but this is not true. If you are logged in to the SME Server shell a special mechanism is in place to log you in with MySQL root privileges without prompting you for the password.
The MySQL root password for SME Server is a 72 character random string generated during installation of SME Server. You should never change the MySQL root password as this will break your SME Server configuration. How to login as MySQL root user? describes how to access MySQL with root privileges on SME Server.
Login as MySQL root user
To login as MySQL root user, simply type 'mysql' at the SME Server shell, this will log you in with root privileges.
Resetting the MySQL root password
To reset the password for the MySQL root account. The MySQL root user on SME Server has a random generated password which is generated during installation. You do not need to know this password to login to MySQL with root privileges on SME Server. If you might have changed the MySQL root password you can reset it like this after getting command line access as root user.
systemctl stop mariadb expand-template /root/.my.cnf expand-template /var/lib/mysql/set.password /usr/libexec/mysqld --socket=/var/lib/mysql/mysql.sock --bootstrap --user=mysql --skip-grant-tables < /var/lib/mysql/set.password exit systemctl start mariadb
cd /var/service/mysqld sv d . expand-template /root/.my.cnf expand-template /var/service/mysqld/set.password /usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < ./set.password sv u .
For SME Server 7.2 and earlier releases do the following (they use the runsvctrl command instead of the sv command):
cd /var/service/mysqld runsvctrl d . expand-template /root/.my.cnf expand-template /var/service/mysqld/set.password /usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < ./set.password runsvctrl u .
Restoring accidentally deleted MySQL root user
mariadb 5.5 and up to 10.5 systemctl stop mariadb echo "GRANT ALL PRIVILEGES ON *.* TO 'root'@'`config get DomainName`' WITH GRANT OPTION;">/var/lib/mysql/set.rootuser echo "GRANT PROXY ON @ TO 'root'@'`config get DomainName`' WITH GRANT OPTION;">>/var/lib/mysql/set.rootuser echo "GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION;">>/var/lib/mysql/set.rootuser echo "GRANT PROXY ON @ TO 'root'@'localhost' WITH GRANT OPTION;">>/var/lib/mysql/set.rootuser expand-template /root/.my.cnf expand-template /var/lib/mysql/set.password /usr/libexec/mysqld --socket=/var/lib/mysql/mysql.sock --bootstrap --user=mysql --skip-grant-tables <( cat /var/lib/mysql/set.rootuser /var/lib/mysql/set.password) exit systemctl start mariadb
for MySQL 5.1.73
cd /var/service/mysqld
sv d .
echo 'use mysql;'>set.rootuser
echo "INSERT INTO `user` VALUES ('localhost','root',,'Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y',,,,,0,0,0,0);">>set.rootuser
expand-template /root/.my.cnf
expand-template /var/service/mysqld/set.password
/usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < set.rootuser
/usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < set.password
sv u .
Note: The following is only applicable on SME 7.3 and MySQL 4.1
cd /var/service/mysqld
sv d .
echo 'use mysql;'>set.rootuser
echo -n 'INSERT INTO user VALUES("localhost","root","",'>>set.rootuser
echo '"Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","","","","",0,0,0);'>>set.rootuser
expand-template /root/.my.cnf
expand-template /var/service/mysqld/set.password
/usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < set.rootuser
/usr/libexec/mysqld --bootstrap --user=mysql --skip-grant-tables < set.password
sv u .
MariaDB/MySQL fails to start
you need to investigate the cause by inspecting two logs :
- service log
journalctl -u mariadb
- mariadb log
tail -f /var/log/mariadb/mariadb.log
Corrupted user table
Your error in mariadb log will include
ERROR: 130 Incorrect file format 'user'
This could mostly occurs after a power outage. mysql.user table is a MYSIAM type
# ll /var/lib/mysql/mysql/user.* -rw-rw---- 1 mysql mysql 10630 3 jui 21:08 /var/lib/mysql/mysql/user.frm -rw-rw---- 1 mysql mysql 488 3 jui 21:08 /var/lib/mysql/mysql/user.MYD -rw-rw---- 1 mysql mysql 2048 3 jui 21:08 /var/lib/mysql/mysql/user.MYI
In this case you might see user.MYD or user.MYI with 0 byte size. If the issue is on MYI this is the index you should be able to rebuild, if it is on the MYD, this is the data, you will need a backup to restore from.
as root, first start mariadb without grant table
systemctl stop mariadb /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --skip-grant-tables
then use mysql command line
mysqlcheck mysql
if wound any error try
mysqlcheck mysql --repair
if it fails then you needs to do a restore. You might have a dump in /home/e-smith/db/mysql/mysql.dump. Wishing it is up to date. I suggest you to copy it and just extract the part for the table you are missing. You need what is under
-- -- Table structure for table `user` --
and
-- -- Dumping data for table `user` --
Considering your table dump is now in a file called /home/e-smith/db/mysql/mysql.user.dump, do
mysql mysql < /home/e-smith/db/mysql/mysql.user.dump expand-template /var/lib/mysql/set.password mysql mysql < /var/lib/mysql/set.password mysqladmin shutdown systemctl start mariadb
Access MariaDB/MySQL using port from the localhost and local network
MariaDB/MySQL on SME Server runs on a socket instead of on a port. MariaDB/MySQL on SME Server is by default configured to allow only localhost connections to improve security, this means that it is only accessible from the server itself and not from the local network nor from the internet. If you wish to enable local network access, execute the following commands on a SME Server shell as root (note access private is not needed as this is the default, and TCPPort 3306 neither as all ports are open to the LAN by default):
config setprop mariadb LocalNetworkingOnly no expand-template /etc/my.cnf systemctl restart /service/mysqld
config setprop mysqld LocalNetworkingOnly no expand-template /etc/my.cnf sv t /service/mysqld
Access MySQL from a remote network
If you wish to enable access to MariaDB/MySQL databases from remote networks, then in addition to the LocalNetworkingOnly db setting mentioned above, you will need to execute the following commands:
config set mariadb service access public status enabled TCPPort 3306 signal-event remoteaccess-update signal-event smeserver-mysql-update
config set mysqld service access public status enabled TCPPort 3306 signal-event remoteaccess-update signal-event reboot
Keep in mind this enables access to your MariaDB/MySQL database for ANYONE, so make sure you have strong passwords on ALL your MariaDB/MySQL databases. Alternatively it would be a more secure approach to require external (remote) users to establish a VPN connection and effectively become part of the local network. In that case do not change the mysql access to public status using the above command.
Create MariaDB/MySQL user(s) with access from other computers
SME Server's default MariaDB/MySQL database users, and most of the database examples in the wiki, allow login only from localhost.
If you want to access a MariaDB/MySQL database on your SME server from other computers, you must not only make the configuration changes described above, you must also create a user who is allowed to login from those systems (see 5.5.4. Access Control, Stage 1: Connection Verification for more detail).
Allow mysql login from any LAN workstation
Assuming your local network is 192.168.1.0, you can create a user with MariaDB/MySQL access from any LAN workstation (or VPN client) using the command shown below (courtesy of DarkMirage).
You probably want to change:
- the database name (MyDB)
- the user name (MyUser)
- the password (MyPW) and
- the allowed computers (192.168.1.%)
## In the command below, \ escapes a linebreak. ## Either include them, or place the entire command on one line mysql -e "\ create database MyDB; \ GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,ALTER \ ON *.* \ TO 'MyUser'@'192.168.1.%' \ IDENTIFIED BY 'MyPW'; \ FLUSH PRIVILEGES;"
Security Implications of allowing remote MariaDB/MySQL login
It is technically possible to combine the above techniques to allow remote MariaDB/MySQL login from any host on the Internet (allow network login, open the firewall, then set the network address to '%'). This would be a bad idea.
If you have remote users who need access to your MariaDB/MySQL database(s), encourage them to use a VPN connection, or an SSH tunnel, or (at a minimum), restrict the allowed login hosts to their internet IP address. On top of that, you are encouraged to enforce encrypted connection at the level of you MariaDB/MySQL service to avoid any clear text exchange on the LAN or worse on the Internet.
Enable InnoDB engine
To enable the InnoDB engine, run the following commands:
db configuration setprop mysqld InnoDB enabled expand-template /etc/my.cnf sv t /service/mysqld
To disable the InnoDB engine, run the following commands:
db configuration setprop mysqld InnoDB disabled expand-template /etc/my.cnf sv t /service/mysqld
Administration
Information about user managament can be found in the MySQL User Account Management section of the MySQL manual, which holds a lot of useful information, a small section is listed here for convenience.
Create a new database
- See the developers guide if you wish to automate the creation of a database within an rpm
or
- Get access to the SME Server shell and issue the following commands:
mysqladmin create 'dbname' --default-character-set=utf8
This will create an empty database called dbname.
Creating MySQL user(s)
Decide which permissions you will have to give to the user on what database. Details about this can be found in the MariaDB/MySQL Manual found at the MariaDB/MySQL site. Get access to the SME Server shell and issue the following commands to login to the MySQL server:
mysql
Suppose we want to create a user which has read-only access on all tables in the database called 'test':
GRANT SELECT ON test.* TO 'user'@'host' IDENTIFIED BY 'password';
In the above line you will have to fill in the user and the host and/or domain from which you will allow the user access to the SME Server MariaDB/MySQL server (don't forget the single quotes). More information can be found in the MariaDB/MySQL Server Manual at the MariaDB/MySQL website linked here.
Listing available databases
To view a list of available databases on the system you can issue the following command while logged in in MariaDB/MySQL:
show databases;
Remove a database
Get access to the SME Server shell and MariaDB/MySQL and issue the following command:
drop database databasename;
Replace databasename with the name of the database.
Remove a user
Get access to the SME Server shell and MariaDB/MySQL and issue the following command:
USE mysql; DELETE FROM user WHERE user = 'username'; FLUSH PRIVILEGES;
Replace username with the username you wish to delete.
Optimizing MariaDB/MySQL default settings for SME 10
Here are the available settings from the configuration database to tweak you MariaDB service. If no default value indicated, please refers the the manual of your database version for its own default value:
| key | default | Role |
|---|---|---|
| innodb_file_format | barracuda | |
| innodb_file_per_table | 1 | |
| LocalNetworkingOnly | no | |
| OpenFilesLimit | 0 | |
| MaxConnections | ||
| WaitTimeout | ||
| QueryCacheLimit | ||
| QueryCacheSize | 1M | |
| QueryCacheType | 1 | |
| SortBufferSize | ||
| ReadRndBufferSize | ||
| TableOpenCache | ||
| TableOpenCacheInstances | ||
| TmpTableSize | ||
| MaxHeapTableSize | ||
| ThreadCacheSize | 256 | |
| KeyBufferSize | key_buffer_size | |
| MyisamSortBufferSize | myisam_sort_buffer_size | |
| JoinBufferSize | 262144 | |
| ReadBufferSize | ||
| MaxConnectErrors | ||
| ConnectTimeout | 100 | |
| MaxAllowedPacket | 16M | |
| SlowQueries |
to alter a value, just do
config set mariadb KeyBufferSize 18M MyisamSortBufferSize 8M expand-template /etc/my.cnf systemctl restart mariadb
if your needed option is not available then create a dedicated template custom. Be careful to use a name starting with a number between 016 and 039.
mkdir -p /etc/e-smith/templates-custom/etc/my.cnf/ vim /etc/e-smith/templates-custom/etc/my.cnf/017myvalues expand-template /etc/my.cnf systemctl restart mariadb
Optimizing MariaDB/MySQL default settings for up to SME9
SME Server uses MariaDB/MySQL for the webmail package, and the default configuration is optimized for that.
If you are using the SME server to provide MariaDB/MySQL databases for functions running on workstations, you may need to adjust some of the default MariaDB/MySQL parameters. Keep in mind it is difficult to optimize MYSQL for a number of different applications, as default values that are suitable for one application may not be suitable for another. In determining appropriate settings for MariaDB/MySQL, you will also need to consider the system resources & general specification of the server that MariaDB/MySQL is running on.
Pointers for tuning and optimizing the databases can be found at http://www.mysqlperformanceblog.com/2006/09/29/what-to-tune-in-mysql-server-after-installation/ and http://lists.mysql.com/mysql/214398 and specifically re key_buffer_size at http://lists.mysql.com/mysql/214398
The following example comes from this forum thread http://forums.contribs.org/index.php/topic,46694.0.html and refers to this bug report http://bugs.contribs.org/show_bug.cgi?id=6287
To change the following parameters
key_buffer_size=18M myisam_sort_buffer_size=8M
Create a custom template fragment & edit it to include your required parameters
mkdir -p /etc/e-smith/templates-custom/etc/my.cnf/ vim /etc/e-smith/templates-custom/etc/my.cnf/016mysetup
Save & Exit
Ctrl o Ctrl x
Expand the changes & restart mysql
expand-template /etc/my.cnf sv t /service/mysqld
Check /etc/my.cnf to see that the changes are reflected.
Is this article helpful to you?
Please consider donating or volunteering
Thank you!
Information on the email subsystem used in SME Server covering sending/recieving, spam filtering, virus checking, webmail, domains and users.
Troubleshooting
I am having trouble getting sme to send and receive email.
Sending and receiving email are separate functions. You need to investigate each individually.
Sending
If SME server does not send mail, you need to examine the /var/log/qmail/current logs to see what happens when it tries. Most commonly problems can be solved by sending via your ISP's mail server, possibly using encryption and/or authentication. Read the manual.
Receiving
If SME server does not receive mail, then you need to ensure that SMTP connections reach your SME server (DNS settings, router configuration, ISP port blocks) and then you need to examine /var/log/qpsmtpd/current logs to determine what SME server does with the incoming connections. Most problems are DNS, router or ISP issues, and have nothing to do with SME server operation or configuration.
qpsmtpd "Connection Timed Out" errors
See Bugzilla:6888 and Bugzilla:2360
A qpsmtpd timeout error may arise, this is not an issue that is caused by SME server directly, however it can become an issue depending on hardware and configuration settings that are contained in and around other enviroments.
It is discussed under various names
- Path MTU Discovery Blackhole http://www.phildev.net/mss/mss-talk.pdf
- Path MTU Discovery Failures http://www.wand.net.nz/~mluckie/pubs/debugging-pmtud.imc2005.pdf
- TCP Problems with Path MTU Discovery http://www.ietf.org/rfc/rfc2923.txt
As discussed in Bugzilla:6888 a workaround was found that may help in mitigating the issue.
The tracepath utility (included with SME 8.0 and SME 7.6) can be used to locate non-standard MTU values between your SME server and any remote host.
You can discover the smallest MTU between you and google.com (for example) by running this command, then locating the smallest value of "pmtu" in the results:
tracepath google.com
If tracepath returns any value below 1500 between your SME server and a mail server that you need to receive email from, you may need to reset the MTU on the SME server to match the smallest value returned.
For example, if tracepath returns 1492 (typical for internet connections using PPPoE), you would need to set the MTU on your SME server to the same value (1492) using the following:
config setprop InternalInterface MTU 1492 signal-event post-upgrade; signal-event reboot
Webmail broken after upgrade
After the usual post-upgrade and reboot, webmail is broken with messages like the following in the messages log:
Apr 20 17:29:53 mail [4614]: PHP Fatal error: Call to a member function on a non-object in /home/httpd/html/horde/imp/lib/Block/tree_folders.php on line 65 Apr 20 17:29:53 mail [4614]: PHP Warning: Unknown(): Unable to call () - function does not exist in Unknown on line 0
As workaround, logout of Horde, close the browser, reopen, log in to Horde, Webmail should now be fully functional. (Based on suggested fix in Bugzilla:5177)
Spam
Spamassassin
Spam filter with Server-Manager
Using the Server-Manager Configuration/E-Mail panel, adjust the settings to these reasonable defaults.
- Virus scanning Enabled
- Spam filtering Enabled
- Spam sensitivity Custom
- Custom spam tagging level 4
- Custom spam rejection level 12
- Sort spam into junkmail folder Enabled
- Modify subject of spam messages Enabled
I would also recommend blocking all executable content. To do so, select (highlight) all of the attachment types other than zip files (the last two).
Click Save.
How It Works
When receiving an incoming message, the server first tests for RBL and DNSBL listings, if enabled. If the sender is blacklisted, the messages are blocked outright and Spamassassin never sees it.
With this configuration, the spammiest messages, those marked as 12 or above, will be rejected at the SMTP level. Those spam messages marked between 4 and 12, will be routed to the users' (IMAP) junkmail folder. This is done so the users can check for false-positives...valid messages that were classified as spam by SpamAssassin.
Users may check their junkmail folders for false-positives via webmail, or, if they are using an IMAP mail client, by simply checking the junkmail folder exposed by their mail client.
https://servername/webmail
Enable/Disable Filtering Per-User
This procedure doesn't really disable the spam filtering, it just stopps the spam from being routed to the 'junkmail' folder.
Per-user filtering is enabled by default. Disable filtering with the following command, as root:
db accounts setprop USERNAME SortSpam disabled db accounts show USERNAME # only displays settings signal-event user-modify USERNAME
Use the Junkmail folder
The Default spamassassin behaviour put spams in the inbox which is very convenient for users in case of false positive, but it is not practical for learning, and especially it does not facilitate the life of the user (setting is available via the manager). If you want to put directly spams in the junkmail folder issue the command above.
config setprop spamassassin SortSpam enabled signal-event email-update
Message Retention Time
Set spamassassin for automatically delete junkmail. You can change the "days" that spamassassin sets to automatically delete junkmail, to delete after two months
db configuration setprop spamassassin MessageRetentionTime 60 signal-event email-update
Spam score Level and Spam score rejection
The "Custom spam rejection level" will only work when "Spam sensitivity" is set to custom.
- Open server-manager.
- Click e-mail in the navigation pane (left-hand side).
- Click Change e-mail filtering settings.
- Change "Spam sensitivity" to custom and adjust the settings to your liking.
This happens because by default, no mail (except for viruses) gets rejected without the admin doing something first.
As a reference, the following setting will have the following behaviours :
| Sensitivity | Spam tagging level | Spam rejection level |
|---|---|---|
| Custom | TagLevel value (Custom spam tagging level) |
RejectLevel value (Custom spam rejection level) |
| veryhigh | 2 | No rejection |
| high | 3 | No rejection |
| medium | 5 | No rejection |
| low | 7 | No rejection |
| verylow | 9 | No rejection |
X-Spam-Level Header in Email Messages
SME does not create an X-Spam-Level header in processed email messages by default.
To enable this capability:
/usr/bin/yum install --enablerepo=smecontribs smeserver-qpsmtpd-spamassassinlevelstars signal-event email-update
(Based on Bugzilla:3505)
spamassassin qpsmtpd's plugins email size limit
This db configuration setting sets the maximum email size above which spamassassin will not apply the spam filtering rules as have been set.
The default setting is 500kb, to increase the maximum size, apply the following commands from a root terminal
db configuration setprop spamassassin MaxMessageSize 2000000
increases message size to 2mb, apply the change with
signal-event email-update
(Based on Bugzilla:7606)
Custom Rule Scores
You can customize the score assigned by a specific Spamassassin rule (SARE_ADULT2 in this case) as follows:
mkdir -p /etc/e-smith/templates-custom/etc/mail/spamassassin/local.cf cd /etc/e-smith/templates-custom/etc/mail/spamassassin/local.cf echo "score SARE_ADULT2 20.000" >> 20localscores signal-event email-update
You can now add additional tests and custom scores by editing the newly-created template fragment 20localscores and adding new custom scores using:
nano -w /etc/e-smith/templates-custom/etc/mail/spamassassin/local.cf/20localscores signal-event email-update
Each custom score goes on its own line. If you enter a score surrounded by parentheses, the "custom" score will be added to the default score for the specified test (use score TEST_NAME (-1) to reduce the score for 'TEST_NAME' by 1)
You can remove these customizations using:
rm -f /etc/e-smith/templates-custom/etc/mail/spamassassin/local.cf/20localscores signal-event email-update
References:
- http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Conf.html#scoring_options
- http://spamassassin.apache.org/tests_3_2_x.html
- http://www.rulesemporium.com/
SPF mail rejection/flagging policy
SME server can protect based of SPF records using spamassassin and the 'sender_permitted_from' plugin. The following lines will enable the plugin.
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ cd /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ echo sender_permitted_from spf_deny 1 > 30spf /sbin/e-smith/expand-template /var/service/qpsmtpd/config/peers/0
Then set your custom rule scores using the Custom Rule Scores section of this page. You should base these scores on your settings in server-manager > Configuration > Email > Change e-mail filtering settings or via db config commands for those with that skillset
echo "score SPF_SOFTFAIL 6.000" >> 20localscores echo "score SPF_FAIL 14.000" >> 20localscores signal-event email-update
In our testing an email that doesn't match SPF records and the sender domain owner has defined a soft fail, if is attributed 6 points and sorted to junkmail folder. If the sender domain owner has defined a hard fail the email attibuted 14 points and is subsequently rejected.
References (but instructions changed to meet new qmail structure):
Pyzor Timeout
See Bugzilla: 5973
This can be mitigated by the adding of a template fragment.
Template fragment to set a pyzor_timeout based on a value in the config db. If no value is set, there is no output (so pyzor uses it's internal default).
mkdir -p /etc/e-smith/templates/etc/mail/spamassassin/local.cf/50pyzor_timeout cd /etc/e-smith/templates/etc/mail/spamassassin/local.cf/50pyzor_timeout nano 50pyzor_timeout
Contents of 50pyzor_timeout
{
my $pyzor_timeout = ($spamassassin{PyzorTimeout} || 0);
if ($pyzor_timeout ne '0')
{
return "pyzor_timeout " . ($pyzor_timeout);
}
}
Then a value can be set using:
config setprop spamassassin PyzorTimeout 15 signal-event email-update
Whitelist and Blacklist
If mail comes in and it is misclassified as spam by Spamasassin, you can add the sender to the Spamassassin whitelist so that future messages coming in from that sender are not filtered. Conversely, you can add a spammer to the Spamassassin blacklist so you never see their spam again. Add senders (or their entire domains) to the global whitelist (or blacklist) with commands similar to these (as root):
db spamassassin setprop wbl.global *@vonage.com White db spamassassin setprop wbl.global *domain2.com White db spamassassin setprop wbl.global user@domain3.com White db spamassassin setprop wbl.global spammer@spamdomain.com Black
you can block an entire TLD but please be aware that you might be denying a legitimate email in the future.
db spamassassin setprop wbl.global *@*.xyz Black db spamassassin setprop wbl.global *@*.link Black
expland template and save the configuration to the database
signal-event email-update
You can view the lists with this command:
db spamassassin show
These lists can be also controlled by the server-manager with the wbl contrib http://wiki.contribs.org/Email_Whitelist-Blacklist_Control
Testing
You can check the auto-learning statistics with this command. You will be able to note the accumulation of the spam tokens (or not). Note that the Bayesian filtering must receive 200 spam messages before it starts to function, so don't expect instantaneous results.
sa-learn --dump magic
You can check the spam filter log with this command:
tail -50 /var/log/spamd/current | tai64nlocal
Check spamassassin configuration like this:
spamassassin -D --lint
If you ever see an error such as:
warn: bayes: cannot open bayes databases /etc/mail/spamassassin/bayes_* R/W: tie failed: Permission denied
Try adjusting some permissions with these commands:
chown :spamd /var/spool/spamd/.spamassassin/* chmod g+rw /var/spool/spamd/.spamassassin/*
Real-time Blackhole List (RBL)
Enabling RBL's
RBL's are disabled by default to allow maximum accommodation (your ISP may be on a RBL & you may not know it). You can enable RBL's by:
config setprop qpsmtpd DNSBL enabled RHSBL enabled signal-event email-update
You can see your RBL's by:
config show qpsmtpd
You can add to your RBL's by:
config setprop qpsmtpd RBLList <rbl-list-name> signal-event email-update
Many will argue what's best, some say the SME defaults are too aggressive and affect some popular free webmail accounts, but most would agree that you can set stable, conservative and non aggressive settings by:
config setprop qpsmtpd RBLList zen.spamhaus.org signal-event email-update
A conservative setting for the associated DNSBL SBLList is:
config setprop qpsmtpd SBLList dbl.spamhaus.org signal-event email-update
Note: More information on this topic can be found here:
[3]
[4]
Possible issues with RBL
When an external dns provider is set in the console menu, it may interfere with some blacklists activated here (RHSBL and DNSBL). The black.uribl.com is know to bounce all emails in this case with a rejection message delivered to the sender. You can in this case
- Remove the black.uribl.com of your SBLList
config setprop qpsmtpd SBLList multi.surbl.org:rhsbl.sorbs.net:dbl.spamhaus.org signal-event email-update
- Let the SME Server being the only dns resolver by removing the dns provider/forwarder in the console menu.
See http://uribl.com/about.shtml#abuse for more information about this issue with black.uribl.com
Obsolete lists
These lists can not be used with smeserver. A migrate fragment will remove them from your settings each time you reconfigure your server.
- RBLList
combined.njabl.org
list.dsbl.org
multihop.dsbl.org
dnsbl.ahbl.org
- SBLLIST
blackhole.securitysage.com
bulk.rhs.mailpolice.com
fraud.rhs.mailpolice.com
porn.rhs.mailpolice.com
adult.rhs.mailpolice.com
bogusmx.rfc-ignorant.org
ex.dnsbl.org
Server Only
Some of the spam filter rules cannot work unless the SMESERVER knows the external IP of the box. If you put a SMESERVER in server-only mode behind other firewalls, it will lose some of the anti-spam rules. For example, the rule that blocks attempts where spammers try "HELO a.b.c.d" where a.b.c.d is your external IP address.
Unfortunately, many admins believe that port-forwarding SMTP provides additional security. It doesn't, it limits the SMESERVER's ability to apply some rules.
I want to enable GreyListing
GreyListing support is under the covers and can easily be enabled for those who know what they are doing. However, many experienced users found that they spent more time looking after the greylisting configuration than they received in benefit. see Greylisting
Bayesian Filtering
From Wikipedia:
Naive Bayes classifiers work by correlating the use of tokens (typically words, or sometimes other things), with spam and non-spam e-mails and then using Bayes' theorem to calculate a probability that an email is or is not spam.
SME server supports bayesian filtering, but does not have it enabled by default.
Enabling bayesian filtering, autolearning, and spam/ham training allows spamassassin to learn from received email and improve spam filter performance. Bugzilla: 6822
Bayesian Autolearning
The following command will enable the bayesian learning filter and set thresholds for the bayesian filter.
config setprop spamassassin UseBayes 1 config setprop spamassassin BayesAutoLearnThresholdSpam 6.00 config setprop spamassassin BayesAutoLearnThresholdNonspam 0.10 config setprop spamassassin UseBayesAutoLearn 1 expand-template /etc/mail/spamassassin/local.cf sa-learn --sync --dbpath /var/spool/spamd/.spamassassin -u spamd chown spamd.spamd /var/spool/spamd/.spamassassin/bayes_* chown spamd.spamd /var/spool/spamd/.spamassassin/bayes.mutex chmod 640 /var/spool/spamd/.spamassassin/bayes_* config setprop spamassassin status enabled config setprop spamassassin RejectLevel 12 config setprop spamassassin TagLevel 4 config setprop spamassassin Sensitivity custom config setprop spamd SpamLearning enabled signal-event email-update
These commands will:
- enable spamassassin
- configure spamassassin to reject any email with a score above 12
- tag spam scored between 4 and 12 in the email header
- enable bayesian filter
- 'autolearn' as SPAM any email with a score above 6.00
Note: SpamAssassin requires at least 3 points from the header, and 3 points from the body to auto-learn as spam. Therefore, the minimum working value for this option is 6, to be changed in increments of 3, 12 considered to be a good working value..
- 'autolearn' as HAM any email with a score below 0.10
Check the bayes stats with the command:
sa-learn --dump magic
The database is located in /var/spool/spamd/.spamassassin/bayes
LearnAsSpam / LearnAsHam (spam/ham training)
LearnAsSpam & LearnAsHam are scripts that can be installed on your server to allow users to manually "train" the bayes database. Training is done by users moving Spam from their Inbox to the "LearnAsSpam" folder, and by COPYING real email that was delivered to junkmail into the "LearnAsHam" folder. All messages in both LearnAsSpam and LearnAsHam are deleted once they have been processed and their tokens have been added to the bayes database.
To install:
- Enable bayes database as described in Bayesian Autolearning (not the best approach, prefer manual learn by user), or
- Install smeserver-learn as per wiki page Learn(and keep auto-learning off), then
- Instruct your users to move any SPAM they find from their Inbox to their LearnAsSpam folder, and to COPY any non-spam (ham) they find in their junkmail folder into their LearnAsHam folder.
This is a really efficient way to reduce impact of SPAM to your particular installation. Do not fear to run again files that are tagged as SPAM, as they will either get ignored if all their patterns are known, or the Bayes might catch one more pattern that could help you to get ride of the next incoming SPAM to even get accepted.
If you want, the code below counts how many e-mail are in LearnAsSpam and LearnAsHam directories (of all users). It's useful to know if your users are using those folders. However Learn will send you a report after each pass. If you are interested on the number of emails lefts in the junkmail directory without any attention, you could install smeserver-mailstats and activate the option to account for them
#!/bin/bash
# ContaLearn.sh
#for compatibility with older versions without rpm, testing
[ `/sbin/e-smith/db configuration getprop LearnAsSpam dir` ] &&
LearnAsSpam=`/sbin/e-smith/db configuration getprop LearnAsSpam dir` || LearnAsSpam='LearnAsSpam';
[ `/sbin/e-smith/db configuration getprop LearnAsHam dir` ] &&
LearnAsHam=`/sbin/e-smith/db configuration getprop LearnAsHam dir` || LearnAsHam='LearnAsSpam';
JunkMail='junkmail';
echo
date
declare -i tspam
declare -i tham
declare -i tleft
declare -i tnseen
printf "%-25s %-11s %-11s %-11s %-11s \n" "User" "LearnAsSpam" "LearnAsHam" "JunkMail" "NotSeen"
pushd /home/e-smith/files/users/ >>/dev/nul
for u in `ls ` #| grep -v admin`
do
[ "$u" = "admin" ] && mailpath="/home/e-smith/" || mailpath="/home/e-smith/files/users/$u" ;
spam=`ls -1 $mailpath/Maildir/.$LearnAsSpam/cur |wc -l`
ham=`ls -1 $mailpath/Maildir/.$LearnAsHam/cur |wc -l`
left=`ls -1 $mailpath/Maildir/.$JunkMail/cur |wc -l`
nseen=`ls -1 $mailpath/Maildir/.$JunkMail/new |wc -l`
if [[ $spam > 0 ]] || [[ $ham > 0 ]] || [[ $left > 0 ]] || [[ $nseen > 0 ]]; then
printf "%-25s %-11d %-11d %-11d %-11d \n" $u $spam $ham $left $nseen
fi
tspam=$tspam+$spam
tham=$tham+$ham
tleft=$tleft+$left
tnseen=$tnseen+$nseen
done
echo "----------------------------------------------------------------------"
printf "%-25s %-11d %-11d %-11d %-11d \n" "Total:" $tspam $tham $tleft $tnseen
echo
popd >>/dev/nul
Learn Contrib
The Learn contrib is intended to install and configure the bayes training tools LearnAsSpam & LearnAsHam.
Reset the Bayes Database
Based on this forum post http://forums.contribs.org/index.php/topic,50712.msg258844.html#msg258844 it may be advantageous to remove the bayes database every few years & recreate it, in order to improve spam filtering performance.
Follow these instructions to turn bayes OFF, delete the database, create an empty database, and turn bayes back on:
config setprop spamassassin UseBayes 0 signal-event email-update 'rm' /var/spool/spamd/.spamassassin/bayes*
config setprop spamassassin UseBayes 1 expand-template /etc/mail/spamassassin/local.cf sa-learn --sync --dbpath /var/spool/spamd/.spamassassin -u spamd chown spamd.spamd /var/spool/spamd/.spamassassin/bayes_* chown spamd.spamd /var/spool/spamd/.spamassassin/bayes.mutex chmod 640 /var/spool/spamd/.spamassassin/bayes_* signal-event email-update
Updates to smeserver-spamassasin now require two new config db settings to have bayesian autolearning enabled. See forum post https://forums.contribs.org/index.php/topic,54320.msg284208.html#msg284208
The Sonora Communications "Spam Filter Configuration for SME 7" howto
http://www.sonoracomm.com/support/19-inet-support/49-spam-filter-configuration-for-sme-7
GeoIP: spam blocking based on geographical information
The GeoIP plugin for Spamassasin lets us know where our mail server is receiving mail from. If we're receiving too much spam from a particular location, this will help track it down. We can then use that info to reject connections from that place taking the load off our server.
You can find information how to install and use it on the GeoIP page.
Anti Virus
SME Server uses Clam AntiVirus (http://www.clamav.net) as the default and built-in anti virus engine.
Signatures
By default SME Server will automatically get virus signature database updates from ClamAV.
Other people and organizations have developed additional signatures which can also be used with ClamAV to provide extra protection. Databases of these signatures can be downloaded and installed on SME Server, and used by ClamAV
In order to automate the download and installation of the additional databases, as well as control which databases you use, follow the instruction in the Virus:Additional Signatures Howto
Heuristic Scan
HeuristicScanPrecedence is a new option in clamav 0.94.
When enabled, if a heuristic scan (such as phishingScam) detects a possible virus/phish it will stop scan immediately. Recommended, saves CPU scan-time.
To enable this feature:
config setprop clamav HeuristicScanPrecedence yes expand-template /etc/clamd.conf sv t clamd
Default is disabled.
Attachment Filtering
The functionality to block possible executable and virus files attached to emails has been incorporated into SME Server v7.x. See the Email panel in server manager.
Additional file signature patterns can be added to the SME defaults. See the Virus:Email Attachment Blocking Howto for further information
Email Clients
"concurrency limit reached" when using IMAP
Sometime shows as Thunderbird giving this error message, This Mail-server is not a imap4 mail-server
To workaround thunderbirds limitations change, this thunderbird setting to false
- Preferences, Advanced, Config editor (aka about:config): filter on tls.
- set security.enable_tls to false
If the total concurrency limit is reached, it'll look like this in /var/log/dovecot/current:
@400000005a1c2c1f19c9381c master: Warning: service(imap): process_limit (2) reached, client connections are being dropped
@400000005a1c2c291a4712dc imap-login: Error: read(imap) failed: Remote closed connection (destination service { process_limit } reached?)
@400000005a1c2c291a471aac imap-login: Error: read(imap) failed: Remote closed connection (destination service { process_limit } reached?)
For the per IP concurrency limit, it'll be like this:
@400000005a1c2c6214542b94 imap-login: Info: Maximum number of connections from user+IP exceeded (mail_max_userip_connections=2): user=<someone>, method=PLAIN, rip=192.168.x.y, lip=192.168.z.t, TLS, session=<abcdefgh>
@400000005a1c2c6233f1bcb4 imap-login: Info: Maximum number of connections from user+IP exceeded (mail_max_userip_connections=2): user=<someone>, method=PLAIN, rip=192.168.x.y, lip=192.168.z.t, TLS, session=<ijklmnop>
The following commands will give your the current value:
db configuration getprop imap ConcurrencyLimit || echo 400 db configuration getprop imap ConcurrencyLimitPerIP || echo 12
You can also increase the ConcurrencyLimitPerIP and/or ConcurrencyLimit value for imap and/or imaps (secure)
config setprop imap ConcurrencyLimitPerIP 20 config setprop imaps ConcurrencyLimitPerIP 20 signal-event post-upgrade; signal-event reboot
To see configuration:
config show imap
tail -f /var/log/dovecot/current | tai64nlocal #out of date
More detail can be found here or here.
Mail server is not an IMAP4 mail server
This is a bug in Thunderbird, the previous tips may help.
The Bat
The gives this error message, but they are wrong.
"This server uses TLS v3.0 which is considered to be obsolete and insecure.
The server must use TLS v3.1 or above."
Outlook/Outlook Express give error 10060/0x800CCC90
Most likely OUTLOOK (EXPRESS) isn't configured correctly.
-open OUTLOOK -click TOOLS > ACCOUNTS -click CHANGE (on the right-hand side) -find INCOMING MAIL SERVER & OUTGOING MAIL SERVER (on right-hand side) -type: mail.yourdomain.tld (in both places) -click MORE SETTINGS (on bottom-right) -click OUTGOING SERVER tab (at the top) -checkmark "MY OUTGOING SERVER REQUIRES AUTHENTICATION" -bullet "USE SAME SETTINGS AS INCOMING MAIL SERVER" -click ADVANCED tab (at the top) -find OUTGOING SERVER -checkmark "THIS SERVER REQUIRES A SECURE CONNECTION" (under outgoing server) -change 25 to 465 -[possibly required, secure IMAP is 993] -click OK > NEXT > FINISHED -you're finished, your email should work now
Outlook 2013 on Windows 10 gives "An unknown error occurred, error code 0x8004011c" when attempting an IMAP connection for a DOMAIN user
This is a known issue with the above combination of Windows and Outlook version as of 2015-02-18 (see: Bug 9618).
The following registry key resolves the issue: To work around this problem, set the value of the ProtectionPolicy registry entry to 1 to enable local backup of the MasterKey instead of requiring a RWDC in the following registry subkey:
[HKEY_LOCAL_MACHINE\Software\Microsoft\Cryptography\Protect\Providers\df9d8cd0-1501-11d1-8c7a-00c04fc297eb] "ProtectionPolicy"=dword:00000001
The PortectionPolicy entry may need to be created
Outlook 2013 on Windows 8.1 gives error 0x800CCC1A when sending over SMTP port 465
This is a known issue with the above combination of Windows and Outlook version as of 2015-02-18 (see: Bug 8854).
The following client-side workaround has been suggested on the dovecot mailinglist:
Disable TLS1.2 on the Windows 8.1 client, using a registry entry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS1.2\Client] "DisabledByDefault"=dword:00000001 "Enabled"=dword:00000000
If the registry entry above does not exist on your system, you will have to create it manually.
Whether this is OpenSSL or Microsoft's "fault" is currently not answered.
Outlook test message doesn't come through
You clicked the TEST ACCOUNT SETTINGS in OUTLOOK didn't you? This is a bug in OUTLOOK. The test message sends a test email with 'no Date header'. As the name suggests, this means a message without any date. Since the server doesn't accept mail with 'no Date header' (because it's required) the message is rejected. To test, send an actual message from OUTLOOK.
If you want, you can try THUNDERBIRD. It's like OUTLOOK but made by a different company. It's completely free and works very well at home and at the office.
I can't receive/send email from my application (ACT!, vTiger, MS Outlook, etc)
Most likely, this is a bug the application you're using and not a problem with the SMESERVER. The application sends an email with 'no Date header'. As the name suggests, this means a message without any date. Since the server doesn't accept mail with 'no Date header' (because it's required) the message is rejected.
As a workaround you can disable the check for the 'Date header'. To disable this check on the internal interface:
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local cd /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local echo "# 17check_basicheaders disabled by custom template" > \ 17check_basicheaders signal-event email-update
To disable this check for the external interface:
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 cd /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 echo "# 17check_basicheaders disabled by custom template" > \ 17check_basicheaders signal-event email-update
After I upgrade my SME Server, my email folders have disappeared when using IMAP
After upgrade, if there are missing IMAP folders, the client may need to re-subscribe to folders. This may affect either webmail users or users who use an IMAP email client.
Entourage: Using SME's Self-Signed Certificate for SSL Connections from Entourage on OS X 10.4
The main problem here is that Entourage will only support trusted, PEM Base-64 Encoded certificates. To use IMAPS or SMTPS from Entourage with your SME server, you will need to:
1. Login to your Mac as a user with administrative privileges 2. Open Safari and browse to https://smeserver/server-manager. When you receive the warning about your certificate: - click on "Show Certificate" - click and drag the gold-rimmed image of a certificate to your desktop. You will now have myserver.mydomain.tld.cer on your desktop. 3. Locate and open the Microsoft Cert Manager - "Import" the certificate you downloaded in step 2. 4. Highlight the imported certificate and "Export" it. - Select the "PEM..." format - add "pem." to the beginning of the filename - export it to your Desktop 5. Double-click on the new pem.myserver.mydomain.tld.cer - Apple's Keychain Access application will open. - Select the X509Anchors Keychain and click "OK" 6. While still in Apple's Keychain Access, select the "Certificates" category - Drag pem.myserver.mydomain.tld.cer into the certificates window.
You should now be able to connect to your SME from your Entourage using IMAPS.
If you are accessing your SME server using a different name than the one encoded in the certificate you will still receive a security warning from Entourage, but "OK" will now grant access to your folders.
Notes:
- Procedure mostly taken from http://www.kerio.com/manual/kmsug/en/ch09s06.html
- I still get various other IMAP errors due, I suspect, to the "concurrency limit reached" issue.
- Click on "Show Keychains" in Apple's "Keychain Access" if you need to delete a certificate and try again.
How do I get my e-mail to show the correct From Address
The From address on an e-mail is not supplied by the server. It is supplied by the e-mail client.
- Configure your Account in your e-mail client with the correct FROM address.
- You can change the FROM address in webmail with the following:
- Login to webmail as the user, go to options-personal information and change the identity to have the correct FROM address. You can have multiple identities with a single user.
Some system generated email is created by the server, some contribs may send mail externally, in these cases you need a valid domain name for the server, buy one or use a free provider like dyndns.org
Outlook 365 / Outlook 2019 IMAP Configuration
Microsoft has disabled the ability to enter the IMAP/SMTP username in the account setup wizard in Outlook 365 / 2019 for Windows. The wizard used within Outlook requires that the IMAP/SMTP username be the full email address.
To work around this issue, setup the account using "Mail (Microsoft Outlook 2016)" in the Windows control panel:
Server Settings
qmail ConcurrencyLocal
The default value for /var/qmail/control/concurrencylocal is 20. This setting controls the maximum amount of simultaneous local deliveries.
There is a optional database property (does not show unless changed from the default setting) called ConcurrencyLocal for qmail in the config database. The ConcurrencyLocal property changes the value stored in /var/qmail/control/concurrencylocal.
It can be set, for example to decrease the local concurrency limit
config setprop qmail ConcurrencyLocal 6 signal-event email-update
qmail ConcurrencyRemote
The default value for /var/qmail/control/concurrencyremote is 20. This setting controls the maximum amount of simultaneous remote deliveries.
There is a optional database property (does not show unless changed from the default setting) called ConcurrencyRemote for qmail in the config database. The ConcurrencyRemote property changes the value stored in /var/qmail/control/concurrencyremote.
It can be set, for example to decrease the remote concurrency limit
config setprop qmail ConcurrencyRemote 10 signal-event email-update
Refer also this comment by CB
http://forums.contribs.org/index.php/topic,50091.msg251320.html#msg251320
How long retry before return e-mail as undeliverable
To configure how long SME server will try to delivery a message before return a permanent error
mkdir -p /etc/e-smith/templates-custom/var/qmail/control echo 172800 > /etc/e-smith/templates-custom/var/qmail/control/queuelifetime expand-template /var/qmail/control/queuelifetime sv t qmail
The default value is 604800 seconds, or one week.
The example above shows 172800 seconds, or two days (a weekend for infra upgrade!)
source: http://forums.contribs.org/index.php/topic,47471.0.html
Double bounce messages
To stop admin receiving double bounce messages
config setprop qmail DoubleBounceTo someoneuser signal-event email-update
Or just delete them. You risk losing legitimate double bounces (which are rare, but you want to look at them when they do occur)
config setprop qmail DoubleBounceTo devnull signal-event email-update
see a longer explaination here
Keep a copy of all emails
You may need to keep a copy of all emails sent to or from your email server. This may be for legal, or other reasons.
The following instructions will create a new user account (default is maillog) and forward every email that goes through your SME server to it.
First, log onto the server-manager and create the user maillog
Go to the SME Command Line (logon as root) and issue the following commands:
config setprop qpsmtpd Bcc enabled signal-event email-update
Optionally make the forwarding of the emails invisible to the end user. Without it, there will be an X-Copied-To: header in each email. Run this command before the signal-event
config setprop qpsmtpd BccMode bcc
If you want to view the emails, point your email client at the SME and log on as maillog.
You can modify the default user:
config setprop qpsmtpd BccUser someuser
Keep a copy of outgoing emails only
In addition to the commands in the previous section we will also have to create a custom template as follows:
Log in as root or a user with root privileges
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ cp /etc/e-smith/templates/var/service/qpsmtpd/config/peers/0/13bcc /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ cd /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ nano -w 13bcc
change the code to:
{
return "# bcc disabled" unless ($qpsmtpd{Bcc} eq "enabled");
return "bcc mode " . $qpsmtpd{BccMode} . " outgoing " . $qpsmtpd{BccUser};
}
Save by pressing Ctrl x at the same time and confirm with y
Then enable the changes with
signal-event email-update
More info:
perldoc /usr/share/qpsmtpd/plugins/bcc
Set Helo hostname
Default is set to the hostname.domain, but sometime you might want to have something else to answer with the same as your reverseDNS. You can do one of the followings to only adjust the helo name:
config setprop smtpd HeloHost mydomainname signal-event email-update
or the following to adjust the way your server will present itself everywhere (httpd, qpsmtd...) This might trigger the generation of new ssl certificate, so use it only if you are sure this is what you want to do.
config set DomainName mydomainname signal-event domain-modify signal-event email-update
Set max email size
- IMPORTANT:
bugzilla: 7876points out that if your system has /var/service/qpsmtpd/config/databytes it should be deleted. (Fixed as of smeserver-qpsmtpd-2.4.0-7.el6.sme.noarch - seebugzilla: 8329).
There are several components involved in sending email on a SME server. Each component has a size limit that may affect an email message that passes through the server.
Be aware that email size is not the same thing as attachment size. Binary attachments to email are encoded using techniques that result in email sizes that can be as much as 30% larger than the original attachment. Most major email clients (Thunderbird, Apple Mail, Outlook) allow you to enable a "message size" column in the message list that will show you the size of your email messages (More).
| Subsystem | Function | Default Limit | Command to change size | Notes |
|---|---|---|---|---|
| qmail | Delivers email to local mailboxes and to remote servers | 15000000 | config setprop qmail MaxMessageSize xx000000 | Value is in BYTES. 15000000 equals approximately 15MB. No value means no limit. |
| clamav | Used to scan emails and attachments | 15M | config setprop clamav MaxFileSize 15M | Value includes human-readable abbreviations. "15M" equals 15 MegaBytes. |
| clamd | Involved in attachment virus scanning | 1400000000 | config setprop clamd MemLimit 1400000000 | May require increase per this forum topic |
| qpsmtpd | The clamav plugin to qpsmtpd is called with a specified size limit. | 25000000 | config setprop qpsmtpd MaxScannerSize xx000000 | Value is in BYTES. Question: does this value override the setting of 'MaxFileSize', or will the smaller value prevail? |
| php | The php maximum file upload size will determine the largest file you can attach to an email message using horde (or any other php email client) | 10M | config setprop php UploadMaxFilesize 10M |
clamav
A note about clamav:
ClamAV includes settings to prevent the scanning of archives that could cause problems if fully expanded; if an attachment cannot be scanned, it will be rejected.
In order for changes to take effect, run:
signal-event email-update
These attributes could result in the rejection of a compressed attachment on a SME server:
- ArchiveMaxCompressionRatio (default 300)
- MaxFiles (default 1500)
- MaxRecursion (default 8)
spamassassin
By default the qpsmtpd 'spamassassin' plugin does not pass any messages over 500,000 bytes to spamassassin for scanning.
To change this behavior:
db configuration setprop spamassassin MaxMessageSize 2000000
increases message size to 2,000,000 bytes. Apply the change with
signal-event email-update
Change Horde Webmail Login Page 'Welcome To' Title
The login page for Webmail defaults to "Welcome to Horde Webmail". In order to change this to something like "Welcome to MyDomain Mail"
config setprop horde Name "MyDomain Mail" signal-event email-update
See also:
Other configurable Horde settings DB_Variables Configuration#Horde_(webmail)
Forum post 31093
Add the admin user as an administrator for Horde
config setprop horde Administration enabled signal-event email-update
Large attachments not displaying in webmail
Due to limits set in the PHP configuration it might be that webmail will not display large attachments (see also bugzilla:3990). The following entries are related to the error and can be found in the log files:
/var/log/messages
Mar 13 00:00:12 box1 httpd: PHP Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 154 bytes) in /home/httpd/html/horde/imp/lib/MIME/Contents.php on line 173
/var/log/httpd/error_log
Allowed memory size of 33554432 bytes exhausted (tried to allocate 0 bytes)
The default MemoryLimit setting in PHP is set to 32M the value can be changed using the commands below replacing XX with the value you desire.
db configuration setprop php MemoryLimit XXM expand-template /etc/php.ini sv t httpd-e-smith
Disable mail to a user from an external network
However, this seems to only affect /var/qmail/control/badrcptto - denying external delivery to your users but allowing outbound emails: http://forums.contribs.org/index.php?topic=40449.5
Can be either a user, pseudonym or group
db accounts setprop groupname/username/pseudonym Visible internal signal-event email-update
If you want to remove
db accounts delprop groupname/username/pseudonym Visible signal-event email-update
- If you need to restrict emails for all users you can perform this command line
db accounts show | awk -F "=" '/\=user/ {print $1}' |while read USER; do db accounts setprop $USER Visible internal; done
signal-event email-update
If you want to remove
db accounts show | awk -F "=" '/\=user/ {print $1}' |while read USER; do db accounts delprop $USER Visible; done
signal-event email-update
I can't receive mail at: user@mail.domain.tld
Add mail.domain.tld as a virtualdomain.
-login to SERVER-MANAGER -click DOMAINS (on the left) -click ADD -type: mail.domain.tld
How do I find out who is logged into webmail and what IP number.
This is logged is in /var/log/messages.
Allow SMTP relay of mail without encryption/authentication
Change the configuration of the system from the default, so that it no longer requires encryption/authentication before allowing relaying of mail.
- For most case, you really want to allow few specific clients on your LAN or trusted networks, this is done by setting a coma separated list of ip this way (replace IP1, IP2, IP3 by valid ips).
config set qpsmtpd UnauthenticatedRelayClients IP1,IP2,IP3 signal-event email-update
- In some case you would have a whole dedicated network with appliances needing to send email without auth, this is done this way
db networks setprop {$network} RelayRequiresAuth disabled
signal-event email-update
- In case you needs are not fulfilled because you need to accommodate a list of remote IP or a sub network of a larger trusted network, you can create a custom template. Here for reference the accepted formats:
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/relayclients # a subnetwork by only using a prefix of full ip echo "10.10.0.">> /etc/e-smith/templates-custom/var/service/qpsmtpd/config/relayclients/80custom # an external ip echo "99.10.1.23" >> /etc/e-smith/templates-custom/var/service/qpsmtpd/config/relayclients/80custom # an external network you control echo "164.163.12.1/30" >> /etc/e-smith/templates-custom/var/service/qpsmtpd/config/relayclients/80custom signal-event email-update
- Disable smtp authentication on all local interfaces as shown in
Bugzilla: 6522
config setprop qpsmtpd RelayRequiresAuth disabled signal-event email-update
SMTP Authentication TLS before Auth disable & enable
Since SME v7.5 the default for SMTP Authentication is 'requires TLS before Auth' to increase security. Where a SME7.4 or earlier server with SMTP & SSMTP authentication enabled has been upgraded, users are now unable to send mail. Users will need to enable TLS or Auto for the Authentication encryption setting in their email clients. Some older email clients and devices do not support TLS.
A fix was released in SME7.5.1 to allow this setting to be disabled (ie revert to SME7.4 functionality). Upgrade to SME7.5.1 before using these commands.
To disable this (AUTH without TLS) & revert to SME7.4 defaults do
config setprop qpsmtpd TlsBeforeAuth 0 signal-event email-update
To change back to the sme7.5 & greater default (AUTH with TLS) do
config setprop qpsmtpd TlsBeforeAuth 1 signal-event email-update
See http://forums.contribs.org/index.php/topic,46218.0.html
http://bugs.contribs.org/show_bug.cgi?id=5997
Internet provider's outgoing port 25 is blocked: How to set an alternative outgoing port for the SMTP server
If your Internet provider is blocking outgoing smtp port 25 on your internet connection but your provider is offering an alternative outgoing port (or when using some relay service) you can simply set this alternative port by adding it to the 'Address of Internet provider's mail server' value in the 'E-mail delivery settings' screen of the server-manager like this:
<internet providers mail server name or ip-address>:<alternative port>
For example: mail.mydomain.com:587
This setting does not alter the incoming smtp mail server port on SME server, which will still use port 25. Refer to a workaround in http://wiki.contribs.org/PortRedirect
How do I enable and configure a disclaimer in email messages
A disclaimer message can be added to the footer of all outgoing email messages.
The message can be the same for all domains or it can be different for all domains.
This functionality is part of sme7.2 release so make sure you have upgraded before doing this.
To create a general disclaimer for all domains on your sme server
config setprop smtpd disclaimer enabled nano -w /service/qpsmtpd/config/disclaimer
Enter the required disclaimer text
To save & exit
Ctrl o Ctrl x
To make the changes take effect
signal-event email-update
To create domain specific disclaimers, create seperate domain based disclaimer text files
Delete the general (all domains) disclaimer file if you have already created it
rm /service/qpsmtpd/config/disclaimer config setprop smtpd disclaimer enabled nano -w /service/qpsmtpd/config/disclaimer_domain1.com.au nano -w /service/qpsmtpd/config/disclaimer_domain2.com nano -w /service/qpsmtpd/config/disclaimer_domain3.org
Enter the required text in each disclaimer file
To save & exit
Ctrl o Ctrl x
After making any changes remember to do
signal-event email-update
Note if you only wish to have a disclaimer for some domains, then only create a disclaimer text file for those domains
Note also the criteria for when a disclaimer is attached
(see http://bugs.contribs.org/show_bug.cgi?id=2648)
eg a disclaimer is added to internal to external messages but not internal to internal messages.
To disable the disclaimer function for all domains on your sme server
config setprop smtpd disclaimer disabled signal-event email-update
Email WBL server manager panel
There is a server-manager contrib to allow GUI control of email white and black lists, detailed in the wiki article: Email_Whitelist-Blacklist_Control.
The panel allows easy configuration of functionality that is built into qmail, qpsmtpd and spamassassin. For more information google for qmail & qpsmtpd, read the spamassassin section in this wiki article and see Email#Default_Plugin_Configuration default qpsmtpd plugin confguration).
There are two main sections, Blacklist and Whitelist, where you can control settings.
Note that there are subtle differences in syntax between whitelist and blacklist entries
Blacklist - Black lists are used for rejecting e-mail traffic
DNSBL status - DNSBL is an abbreviation for "DNS blacklist".
It is a list of IP addresses known to be spammers.
RHSBL status - RHSBL is an abbreviation for "Right Hand Side Blacklist".
It is a list of domain names known to be spammers.
qpsmtpd badhelo - Check a HELO message delivered from a connecting host.
Reject any that appear in badhelo during the 'helo' stage.
qmail badmailfrom - Check envelope sender addresses.
Reject any that appear (@host or user@host) in badmailfrom during the 'mail'
stage.
spamassassin blacklist_from - Any envelope sender of a mail (*@host or user@host) matching an
entry in blacklist_from will be rejected by spamassassin.
Whitelists - White lists are used for accepting e-mail traffic
Whitelists status - White Lists: ACCEPT
qpsmtpd whitelisthosts - Any IP address listed in whitelisthosts will be exempted
from any further validation during the 'connect' stage.
qpsmtpd whitelisthelo - Any host that issues a HELO matching an entry in whitelisthelo
will be exempted from further validation during the 'helo' stage.
qpsmtpd whitelistsenders - Any envelope sender of a mail (host or user@host) matching an
entry in whitelistsenders will be exempted from further validation
during the 'mail' stage.
spamassassin whitelist_from - Any envelope sender of a mail (*@host or user@host) matching an
entry in whitelist_from will be exempted from spamassassin rejection.
How to block email from one address to another address with check_badmailfromto plugin
Enable the check_badmailfromto plugin. Adapted from this Forum post
This is based heavily on the similar check_badmailfrom, but this plugin references both the FROM: and TO: lines, and if they both are present in the badmailfromto config file (a tab delimited list of FROM/TO pairs), then the message is blocked as if the recipient (TO) didn't exist. This is specifically designed to not give the impression that the sender is blocked (good for cases of harassment).
Prior SME9.2 : qpsmtpd check_badmailfromto plugin
To control mail from external locations to internal locations do
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins echo "check_badmailfromto" > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31check_badmailfromto ln -s /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31check_badmailfromto /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/31check_badmailfromto signal-event email-update
To control mail sent from internal locations to internal locations, in addition to the above also do
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local ln -s /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31check_badmailfromto /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local/31check_badmailfromto signal-event email-update
Since SME9.2 : qpsmtpd badmailfromto plugin
remove previous templates, if you are updating
rm /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31check_badmailfromto \ /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/31check_badmailfromto \ /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local/31check_badmailfromto
To control mail from external locations to internal locations do
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins echo "badmailfromto" > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31badmailfromto ln -s /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31badmailfromto /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/31badmailfromto signal-event email-update
To control mail sent from internal locations to internal locations, in addition to the above also do
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local ln -s /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/31badmailfromto /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local/31badmailfromto signal-event email-update
For Qmail
Create and configure the badmailfromto custom template fragment
mkdir -p /etc/e-smith/templates-custom/var/qmail/control/badmailfromto nano -w /etc/e-smith/templates-custom/var/qmail/control/badmailfromto/template-begin
Type in the From and To pairs that you want to stop email delivery for, with a tab between them and a carriage return at the end of the line, with additional pairs on a new line ie
user@bad-domain.com tab user@yourdomain.com enter user@bad-domain2 tab user2@yourdomain enter
Note also that wildcards or blank spaces are not supported
eg
john@aol.com mary@yourdomain bill@yahoo.com paul@yourdomain.com
then save using
Ctrl o Ctrl x
Expand the template to update the /var/qmail/control/badmailfromto config file
expand-template /var/qmail/control/badmailfromto
Restart mail services
signal-event email-update
Redirect mail.domain.net to Webmail
Setup external dns records
Add mail.domain.net in Domains panel in server-manager
db domains setprop mail.dom.ain TemplatePath ProxyPassVirtualHosts ProxyPassTarget http://sme.dom.ain/webmail signal-event remoteaccess-update
where http://sme.dom.ain/webmail is servername.domainname/webmail
E-mail Retrieval
http://wiki.contribs.org/SME_Server:Documentation:Administration_Manual:Chapter13#E-mail_Retrieval
If your ISP does not provide a custom sort field and you experience the following errors occuring when Multidrop is enabled and the "Select Sort Method (for multi-drop)" is set to Default:
fetchmail: warning: multidrop for pop3.mypopserver.com requires envelope option! fetchmail: warning: Do not ask for support if all mail goes to postmaster!
and/or
fetchmail: warning: multidrop for my.isp.domain requires envelope option! fetchmail: warning: Do not ask for support if all mail goes to postmaster!
Set "Select Sort Method (for multi-drop) to 'Received' or 'for'
As described at bugzilla:5602 bugzilla:6483
Domain Authentication
Major mail hosting companies (Google, Yahoo, Microsoft) have made domain-authentication mandatory so as to not mark incoming mail as spam.
To facilitate this support for DomainKeys and DKIM signing needs to be enabled in SME's mail subsystem. These techniques require the adding of records in the DNS zone for the user's domain. The DKIM/DK/SPF/SenderID configuration has to be added to your your DNS server / registrar.
How do I remove an email address from the everyone group
By default, all users are automatically added to the user group "everyone". If you would like to remove a user from this group, connect to the server using SSH or locally log in to the server and issue the commands below. Be sure to substitute the name of the user you want to remove for the word username.
db accounts setprop username EveryoneEmail no signal-event user-modify username
How do I remove an email address from any regular group
By default, all users member of a group "group1" are automatically added as recipients of mail sent to group1@domain. If you would like to remove a user from this group, connect to the server using SSH or locally log in to the server and issue the commands below. Be sure to substitute the name of the user you want to remove for the word username.
db accounts setprop group1 EmailExcludeUsers tom,jack signal-event group-modify group1
If you want to prevent all the user members from another group "group2" from receiving emails addressed to group1@domain while they are also member of group1, you could connect to the server using SSH or locally log in to the server and issue the commands below. Be sure to substitute the name of the user you want to remove for the word username.
db accounts setprop group1 EmailExcludeGroups group2 signal-event group-modify group1
All members of the group will still be member for all other purpose (samba access to ibays as an example)
This behaviour is only available as per e-smith-qmail-2.4.0-7.sme see bug #9540
Change the number of logs retained for qpsmtpd and/or sqpsmtpd
The normal retention is 5 logs for both qpsmptd and sqpsmtpd. This may or may not fit all installations. This information is pulled from bugzilla.
Check your config to see if any change has been made to the default log retention rules. Note there are different rules for qpsmtpd and sqpsmtpd. You have to make changes to both as you require.
config show qpsmtpd
If the KeepLogFiles property isn't listed, the default rules apply. Determine how many logs you would like to keep and apply that to the following example. In the command below, 15 is used to keep 15 qpsmtpd logs.
db configuration setprop qpsmtpd KeepLogFiles 15
Restart multilog with the following.
sv t /service/qpsmtpd/log
Check that your setting saved.
ps aux | grep qpsmtpd | grep multi
Look for the line that ends with /var/log/qpsmtpd and verify the number after n equals your KeepLogFiles property from above.
DKIM Setup - qpsmtpd version<0.96
A plugin has been written and is available in SME
To activate it manually follow the steps below, or download a shell script that will do the server based stuff for you & guide you on the DNS stuff setup_dkim.sh:-
Note: I'd recommend reviewing the script first to make sure you're happy to run it on your system
Create a folder:
mkdir /var/service/qpsmtpd/config/dkimkeys/
Then:
cd /var/service/qpsmtpd/config/dkimkeys/ openssl genrsa -out dkim.private 1024 openssl rsa -in dkim.private -pubout -out dkim.public chown qpsmtpd:qpsmtpd -R /var/service/qpsmtpd/config/dkimkeys/ chmod 0700 dkim.private
For each domain you want to sign:
cp -a dkim.private <fully qualified domain name>.private (less the <> brackets)
Then create a template fragment:
mkdir --parent /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local echo "dkim_sign keys dkim">/etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local/69dkim_sign signal-event email-update
Finally propagate your public key "dkim.public" content (<key text>) to your DNS.
Check with your DNS server / registrar. Something similar to the following should work but it varies depending on provider - replace <fully qualified domain name> with your doman details e.g "mydomain.org" (less the <> brackets):
When extracting the key text from the dkim.public file it's on multiple lines. For the key to work for us in the DNS TXT record we need to exclude the header & footer lines & have just the key text as a single line string (the setup_dkim.sh script provides this info in the format required).
default._domainkey.<fully qualified domain name> IN TXT "k=rsa; p=<key text>; t=y"
With Zonedit the following works within your Zone :
Subdomain : default._domainkey
Type : TXT
Text : "v=DKIM1;k=rsa; p=<key text>; t=y"
If you want to customize the signing you can add parameters to the line in /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/local/69dkim_sign. Parameters and value are separated by a space only.
- keys : "dk" or "domainkeys" for domainkey signature only, "dkim" for DKIM signature only, default "both" (n.b. above template example is dkim ONLY)
- dk_method : for domainkey method , default "nofws"
- selector : the selector you want, default "default"
- algorithm : algorithm for DKIM signing, default "rsa-sha1"
- dkim_method : for DKIM, default "relaxed"
NB: key files can not be defined in parameters, they need to be in /var/service/qpsmtpd/config/dkimkeys/{SENDER_DOMAIN}.private
See also : bugzilla:8251 bugzilla:8252
DKIM Setup - qpsmtpd version >= 0.96
Version 0.96 and above supports DKIM natively without the need for extra plugins.
All you have to do is to enable the DKIM signing and promulgate the DNS TXT entries to support it.
Enable the signing:
db configuration setprop qpsmtpd DKIMSigning enabled signal-event email-update
and then run:
qpsmtpd-print-dns <domain name>
to show the DNS entry(s) required.
Then you have to update your DNS.
also see bugzilla:9694 and https://wikit.firewall-services.com/doku.php/smedev/qpsmtpd_096#documentation
More details are available here
Incoming DKIM checking is also enabled out of the box.
In case you got a problem using the DKIM field provided with your DNS provider /registrar, please first contact them to ensure the problem is not how you try to enter the information. In the likelihood, you got "invalid field" or "too long field" errors and your provider is not able to help you or update its interface, you can generate a shorter DKIM key (with 1024 instead of the default 2048) this way:
cd /home/e-smith/dkim_keys/default mv private private.long mv public public.long openssl genrsa -out private 1024 openssl rsa -in private -pubout -out public chown qpsmtpd:qpsmtpd private chown root:qpsmtpd public chmod 0400 private signal-event email-update qpsmtpd-print-dns
Outbound DKIM signing / SPF / DMARC policy FOR MULTIPLE DOMAINS
The default DKIM key is created in /home/e-smith/dkim_keys/default. To enable DKIM signing for all the domains that you manage:
db configuration setprop qpsmtpd DKIMSigning enabled signal-event email-update
If you want to disable dkim signing for a domain, you can use:
db domains setprop domain.com DKIMSigning disabled signal-event email-update
The default behavior is to use the same key pair for all your domains. But you can create other key pairs for specific domain if you want. For example, if you want to use a specific key pair for the domain.net domain:
cd /home/e-smith/dkim_keys mkdir domain.net cd domain.net echo default > selector openssl genrsa -out private 2048 openssl rsa -in private -out public -pubout chown qpsmtpd:qpsmtpd private chmod 400 private signal-event email-update
Now, the emails using a domain.net sender address will be signed by this new key instead of the default one.
Domain Keys
There is a plugin to check incoming mail has been signed
Please read here for more details : http://bugs.contribs.org/show_bug.cgi?id=4569
Other information
DomainKeys seem to be deprecated in favour of DKIM.
The DomainKeys plugin only CHECKS incoming email. Spamassassin checks for DKIM.
Temporary_error_on_maildir_delivery
In certains cases you have some mailboxes which can't delivery messages and the qmail log say:
deferral: Temporary_error_on_maildir_delivery._(#4.3.0)/
It is probably that your users want to go beyond the upper limit of their quota, so you have to increase it. This could solve their problems.
External Access
Allow external IMAP mail access
There was a deliberate decision to remove non-SSL protected username/password services from the external interface.
to allow unsecure IMAP access
config setprop imap access public signal-event email-update
But before you do this try to use secure IMAP (IMAPS or imap over ssl) with port 993
POP3 & webmail HTTP
I want to set my SMESERVER to allow POP3 (or webmail HTTP) but it's not an option, I only see POP3S (or webmail HTTPS).
The SMESERVER is secure by design. POP3 (or webmail HTTP) is viewed as inadequate security and removed as an option from a standard installation to encourage unknowing administrators to select the 'best practice' option -a secure connection with POP3S, IMAPS, or HTTPS.
You can still set your SMESERVER to allow POP3 settings by:
config setprop pop3 access public signal-event email-update
Allow external pop3 access
Email settings > POP3 server access in SME 7.1 server-manager allows only pop3s protocol for clients outside the LAN. Some email clients (eg The Bat! v3.98.4) won't allow pop3s connections to SME 7.1 because of ssl version conflict. Until this is sorted out, a workaround is to hack SME to allow regular pop3 on the external interface using the following commands.
config setprop pop3 access public signal-event email-update svc -t /service/pop3s
more information bugzilla:2620
Imap
Folders with a dot in name
Email folder names that have a period ('.') in the folder name, will be split into sub-folders. e.g. folder name 'www.contribs.org' is created as
www
contribs
org
Dovecot Idle_Notify
Poor battery consumption issues has been reported with K9-mail on recent Android systems. It is apparent one way of helping this is to modify the imap_idle_notify setting. The default is in Dovecot, and therefore on SME is 2 minutes.
K9 has an idle refresh of 24 mins but it seems with Dovecot defaults at 2 mins it causes lots of wake ups and battery drain.
This is configurable via a config db property.
Default on install
# config show dovecot dovecot=service Quotas=enabled status=enabled
Set dovecot Idle_Notify to 20 minutes
# config setprop dovecot Idle_Notify 20 # config show dovecot dovecot=service Idle_Notify=20 Quotas=enabled status=enabled
Expand template to update *.conf (can also issue a full reconfigure/reboot)
# expand-template /etc/dovecot/dovecot.conf # dovecot -a |grep imap_idle_notify_interval imap_idle_notify_interval = 20 mins
qpsmtpd
SME uses the qpsmtpd smtp daemon.
Official Description
qpsmtpd is a flexible smtpd daemon written in Perl. Apart from the core SMTP features, all functionality is implemented in small "extension plugins" using the easy to use object oriented plugin API.
qpsmtpd was originally written as a drop-in qmail-smtpd replacement, but now it also includes smtp forward, postfix, exim and maildir "backends".
qpsmtpd wiki: http://wiki.qpsmtpd.org
Log watching tool
qplogtail is a script to to monitor /var/log/qpsmtpd/current, see bugzilla:3418
Qpsmtpd for SME versions 9.1 and earlier
Default Plugin Configuration
SME uses the following qpsmtpd plugins to evaluate each incoming email.
SME maintains 2 distinct configurations: one for the 'local' networks (as defined in server-manager::Security::Local networks) and another for 'remote' networks (everyone else).
The default configuration of each plugin is indicated in the 'Default Status' column.
| Plugin | Purpose | Default Status |
|---|---|---|
| hosts_allow | Prohibit more than "InstancesPerIP" connections from any single host (change with 'config setprop smtpd InstancesPerIP'). Allow or deny connections according to the contents of /var/service/qpsmtpd/config/hosts_allow. See hosts_allow SVN code for more details. | enabled |
| peers | Allow different plugin configuration based on the sending computer's IP address. By default SME maintains different configurations for the local networks (in /var/service/qpsmtpd/config/peers/local) and for everyone else (in /var/service/qpsmtpd/config/peers/0) | enabled |
| logging/logterse | Allow greater logging detail using smaller log files. Optionally supports qplogsumm.pl to compile qpsmtpd statistics. | enabled |
| auth/auth_cvm_unix_local | Allow authenticated smtp relay | enabled (remote) disabled (local) |
| check_earlytalker | reject email from servers that talk out of turn | enabled (remote) disabled (local) |
| count_unrecognized_commands | reject email from servers that issue X invalid commands | enabled (remote) disabled (local) |
| bcc | bcc all email to a specific address for archiving | disabled |
| check_relay | Check to see if relaying is allowed (in case the recipient is not listed in one of SME's local domains) | enabled |
| check_norelay | Check to see if the sending server is specifically forbidden to relay through us. | enabled |
| require_resolvable_fromhost | Check that the domain listed in the sender's email address is resolvable | enabled (remote) disabled (local) |
| check_basicheaders | reject email that lacks either a From: or Date: header | enabled |
| rhsbl | Reject email if the sender's email domain has a reputation for disregarding smtp RFCs. | disabled (always disabled for local connections) |
| dnsbl | Reject email from hosts listed in your configured dnsbl servers | disabled |
| check_badmailfrom | Reject email where the sender address is listed in /var/service/qpsmtpd/config/badmailfrom | enabled |
| check_badrcptto_patterns | Reject email addressed to any address matching an expression listed in /var/service/qpsmtpd/config/badrcptto_patterns | enabled |
| check_badrcptto | Reject email addressed to any address listed in /var/service/qpsmtpd/config/badrcptto | enabled |
| check_spamhelo | Reject email from hosts that say 'helo ...' using a value in /var/service/qpsmtpd/config/badhelo | enabled |
| check_smtp_forward | If config show DelegateMailServer or db domains show <domainname> MailServer is set (telling SME to deliver email for all domains or just <domainname> to another server), check_smtp_forward will connect to the specified server and will reject the message outright if the internal mail server would also reject it. | disabled unless an internal mail server is configured. |
| check_goodrcptto | Accept email only if the recipient address matches an entry in /var/service/qpsmtpd/config/goodrcptto. For domains that are configured to use an internal mail server, the entire domain name will be added to .../goodrcptto. | enabled |
| rcpt_ok | Return 'OK' if none of the other host checks has returned 'DENY' (??) | enabled |
| pattern_filter | Reject email according to content patterns (??) | disabled |
| tnef2mime | Convert MS TNEF (winmail.dat) and uuencoded attachments to MIME | enabled |
| disclaimer | Add a configurable disclaimer to email messages | disabled |
| spamassassin | Check email using spamassassin, and optionally reject it completely if the score exceeds a configurable value. | disabled (always disabled for local connections) |
| virus/clamav | Scan incoming email with ClamAV | enabled |
| queue/qmail-queue | Deliver the incoming message to qmail for delivery. | enabled |
Qpsmtpd for SME versions 9.2 and Later
This section has been taken from the notes prepared by the dev who made the changes, the wiki is here.
Here is a list of the plugins in use, and a note of any changes that might have occurred:
- logterse: no change
- tls: no change
- auth_cvm_unix_local: no change
- check_earlytalker: renamed earlytalker
- count_unrecognized_commands: no change
- bcc: no change
- check_relay: renamed relay
- check_norelay: merged into the relay plugin
- require_resolvable_fromhost: renamed resolvable_fromhost
- check_basicheaders: renamed headers
- rhsbl: no change
- dnsbl: no change
- check_badmailfrom: renamed badmailfrom
- check_badrcptto_patterns: doesn't exist anymore, merged with badrcptto
- check_badrcptto: renamed badrcptto
- check_spamhelo: renamed helo
- check_smtp_forward: no change
- check_goodrcptto: no change
- rcpt_ok: no change
- pattern_filter: no change
- tnef2mime: no change
- spamassassin: no change
- clamav: no change
- qmail-queue: no change
Here is a section for each of the new plugins which are installed by default. The ones that have not changed are documented above.
Karma
The karma plugin tracks sender history. For each inbound email, various plugins can raise, or lower the "naughtiness" of the connection (eg, if SPF check passes, if the message is spammy etc...). For each host sending us email, the total number of connections, and the number of good and bad connections is recorded in a database. If a host as more bad than good connections in its history, emails will be rejected for 1 day. 3 settings are available for this plugin:
- Karma (enabled|disabled): Default value is disabled. Change to enabled to use the plugin
- KarmaNegative (integer): Default value is 2.
It's the delta between good and bad connection to consider the host naughty enough to block it for 1 day.
Eg, with a default value of two, a host can be considered naughty if it sent you 8 good emails and 10 bad ones - KarmaStrikes (integer): Default value is 3. This is the threshold for a single email to be considered good or bad.
Eg, with the default value of 3, an email needs at least 3 bad karmas (reaches -3) for the connection to be considered bad.
On the other side, 3 good karmas are needed for the connection to be considered good. Between the two, the connection is considered neutral
and won't be used in the history count
Example:
db configuration setprop qpsmtpd Karma enabled KarmaNegative 3 signal-event email-update
URIBL
The URIBL plugin works a bit like RHSBL, except that it checks domain names found in the body of the email. For each URI identified, the corresponding domain name can be submitted to a BL list (through DNS queries). Two settings are available:
- URIBL (enabled|disabled): Default is disabled. Set this to enabled to use the plugin
- UBLList: (Comma separated list addresses): Default value is multi.surbl.org:8-16-64-128,black.uribl.com,rhsbl.sorbs.net.
This can be the same as RBLList. You can also set bitmask to use for combined lists (in the default value, the bitmask is 8-16-64-128)
Example:
db configuration setprop qpsmtpd URIBL enabled UBLList multi.surbl.org,black.uribl.com signal-event email-update
Helo
Previously, the helo plugin was just checking for some known bad helo hostnames used by spammers (aol.com and yahoo.com). Now, it can check much more than that. This plugin is always enabled and has a single setting:
- HeloPolicy: (lenient|rfc|strict). The default value is lenient.
See https://github.com/smtpd/qpsmtpd/blob/master/plugins/helo for a description of the various tests done at each level
Example:
db configuration setprop qpsmtpd HeloPolicy rfc signal-event email-update
Inbound DKIM / SPF / DMARC
DMARC is a policy on top of DKIM and SPF. By default, SPF and DKIM are now checked on every inbound emails, but no reject is attempted. The dmarc plugin can decide to reject the email (depending on the sender policy). dkim and spf plugins are always enabled. dmarc has two settings:
- DMARCReject (enabled|disabled): Default value is disabled.
If set to enabled, the dmarc plugin can decide to reject an email (if the policy of the sender is to reject on alignment failure) - DMARCReporting (enabled|disabled): Default value is enabled.
If set to enabled, enable reporting (which is the r in dmarc). Reporting is a very important part of the DMARC standard.
When enabled, you'll record information about email you receive from domains which have published a DMARC policy in a local
SQLite database (/var/lib/qpsmtpd/dmarc/reports.sqlite).
Then, once a day, you send the aggregate reports to the domain owner so they have feedback.
You can set this to disabled if you want to disable this feature - SPFRejectPolicy (0|1|2|3|4): Default value is 0. Set the policy to apply in case of SPF failure when the sender hasn't published a DMARC policy.
Note: this is only used when no DMARC policy is published by the sender.
If there's a DMARC policy, even a "p=none" one (meaning no reject), then the email won't be rejected, even on failed SPF tests.
- 0: do not reject anything
- 1: reject when SPF says fail
- 2: reject when SPF says softfail
- 3: reject when SPF says neutral
- 4: reject when an error occurred (like a syntax error in SPF entry) or if no SPF entry is published
- Inbound DKIM checks are only used by DMARC. No reject solely based on DKIM is supported
Example:
db configuration setprop qpsmtpd DMARCReject disabled SPFRejectPolicy 2 signal-event email-update
Outbound DKIM signing / SPF / DMARC policy
Everything is now ready for you to sign your outbound emails, and publish your public key, as well as your SPF and DMARC policy. A default DKIM key is created in /home/e-smith/dkim_keys/default. To enable DKIM signing for all the domain you manage:
db configuration setprop qpsmtpd DKIMSigning enabled signal-event email-update
If you want to disable dkim signing for a domain, you can use:
db domains setprop domain.com DKIMSigning disabled signal-event email-update
The default behavior is to use the same key pair for all your domains. But you can create other key pairs for specific domain if you want. For example, if you want to use a specific key pair for the domain.net domain:
cd /home/e-smith/dkim_keys mkdir domain.net cd domain.net echo default > selector openssl genrsa -out private 2048 openssl rsa -in private -out public -pubout chown qpsmtpd:qpsmtpd private chmod 400 private signal-event email-update
Now, the emails using a domain.net sender address will be signed by this new key instead of the default one.
Publishing your DNS entries
Signing your outbound emails is just part of the process. You now need to publish some DNS entries so everyone can check if the email they receive matches your policy. This part is not to be done on your SME Server, but on your public DNS provider. A script helps you by creating some sample DNS entries already formatted for a bind-like zone file. To use it:
qpsmtpd-print-dns <domain name>
If omitted, the primary domain name is assumed.
Example output:
Here are sample DNS entries you should add in your public DNS The DKIM entry can be copied as is, but others will probably need to be adjusted to your need. For example, you should either change the reporting email adress for DMARC (or create the needed pseudonym) default._domainkey IN TXT "v=DKIM1; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/Qq3Ntpx2QNdRxGKMeKc2r9ULvyYW633IbLivHznN9JvjJIbS54PGIEk3sSxvZSdpTRAvYlxn/nRi329VmcDK0vJYb2ut2rnZ3VO3r5srm+XEvTNPxij5eU4gqw+5ayySDjqzAMEMc5V7lUMpZ/YiqnscA075XiMF7iEq8Quv1y0LokmgwtxzOXEZap34WXlKyhYzH+D""fabF6SUllmA0ovODNvudzvEOanPlViQ7q7d+Mc3b7X/fzgJfh5P9f5U+iSmzgyGctSb6GX8sqsDMNVEsRZpSE3jd2Z33RDWyW21PGOKB/ZrLiliKfdJbd3Wo7AN7bWsZpQsei2Hsv1niQIDAQAB" @ IN SPF "v=spf1 mx a -all" @ IN TXT "v=spf1 mx a -all" _dmarc IN TXT "v=DMARC1; p=none; adkim=s; aspf=r; rua=mailto:dmarc-feedback@domain.net; pct=100"
All you have to do now is publish those records, but do note that there is a point to consider when publishing the default._domainkey DNS record, as produced by the qpsmtpd-print-dns command: if the DNS record includes ;t=y then as per the DKIM specification (RFC4781 section 3.6.1) this means that your "...domain is testing DKIM. Verifiers MUST NOT treat messages from signers in testing mode differently from unsigned email, even should the signature fail to verify. Verifiers MAY wish to track testing mode results to assist the signer."
On the other hand, if no ;t=y is included, then it means you are intending to use DKIM in production mode. It might be a good idea to publish the DKIM DNS record first in testing mode (;t=y included), check how things go and if everything is alright, remove the ;t=y part.
Testing
You can install spfquery:
yum --enablerepo=epel install libspf2 libspf2-progs
Usage (try -help for help):
spfquery -ip=11.22.33.44 -sender=user@aol.com -helo=spammer.tld
Check record via dig
dig -t TXT +short somedomain.co.uk
Load
The loadcheck plugin can temporarily deny inbound emails if your server is overloaded. This plugin is always enabled and has a single setting:
- MaxLoad (int number): Default is 7. If your load is above this value, emails from the outside will be deferred.
Other QPSMTPD Plugins
The following qpsmtpd plugins will work on a SME server, but are either not included or are not configured by default.
| Plugin | Purpose | Default Status |
|---|---|---|
| connection_time | Track the total time for each qpsmtpd connection from 'Accepted connection' through 'click, disconnecting', and output the results to the qpsmtpd log file. | not installed - not clear if this works for SME9.2 (anyone?) |
| GeoIP | Track the geographic origin of incoming email and optionally reject email from specified countries | not installed - does work for SME 9.2 and later. |
Internal or External Mail Servers
SME can be configured as a spam and antivirus filter for one or more "Internal or External" mail servers on a domain-by-domain basis. The mail server specified does not have to be on the same local network as your SME server, & can be hosted on an external site.
Deliver ALL email to a single internal or external mail server
You can set the default delivery location for all domains on your SME server to a single internal or external mail server by setting the mail server address in server-manager::Configuration::E-mail::Change e-mail delivery settings::Address of internal mail server.
Note: Address of internal mail server must be blank if you want any email delivered to the SME server itself.
Deliver email for one domain to an internal or external mail server
You can override the default email delivery destination for individual domains on your SME server (forwarding all email for the specified domain to another server) as follows:
First, create the necessary virtual domains using server-manager::Configuration::Domains::Add Domain.
Then, (assuming your domain is called test.com and the actual mail server is at a.b.c.d issue the following commands:
db domains setprop test.com MailServer a.b.c.d signal-event email-update
A FQDN can also be used for the MailServer property, eg aspmx.l.google.com instead of the IP address a.b.c.d
db domains setprop test.com MailServer aspmx.l.google.com signal-event email-update
Remove the internal or external mail server (and return email delivery for test.com to the default for your SME server) using:
db domains delprop test.com MailServer signal-event email-update
Secondary/Backup Mail Server Considerations
Many people misunderstand the issues of using a secondary or backup mail server (backup MX) to hold your mail before it gets delivered to your SME Server. If you consider putting a backup mail server in place because you are concerned about lost mail because your internet connection may occasionally drop out, think again and consider the issues discussed below.
What is Backup MX
A backup MX is a system whereby through your DNS records you tell other servers on the internet that in order to deliver mail to your domain they first need to try the primary MX record and if they fail to connect they can try to connect to one or more of your listed backup or secondary mail servers. See also http://en.wikipedia.org/wiki/MX_record
The process of delivering email to your SME Server
So lets look at how mail gets delivered without and with a backup mx when your Internet link, ISP or server is down.
Without a backup MX
- The sending mail server cannot connect to your server.
- The sending mail server MUST queue the mail and try again later.
- The mail stays on the sender's server.
- The sender's server resends the mail at a later date.
The requirement to re-queue is a fundamental part of the SMTP protocol - it is not optional. So, if your server is offline due to a link or ISP outage, the mail just stays at the sender's server until you are once again reachable.
With a backup MX
- The sending mail server cannot contact your server.
- The sending mail server sends the mail to your secondary MX.
- The secondary MX queues the mail until your link/server is up.
- The mail is queued on an untrusted third-party mail server (think about confidential mail between your company and some business partner).
- The sending mail server's administrator thinks it has been delivered, according to their logs.
- You have no, or little, visibility over the queued mail.
- When your link comes up, the secondary MX sends the mail on to your server.
- You have added more hops, more systems and more delay to the process.
If you think that a backup MX will protect against broken mail servers which don't re-queue, you can't. Those servers will drop mail on the floor at random times, for example when their Internet link is down.
Those servers are also highly likely to never try your backup MX.
Thankfully those servers are mostly gone from the Internet, but adding a secondary MX doesn't really improve the chances that they won't drop mail destined for your server on the floor.
Backup MX and SPAM Filtering
On top of the issue, indicated above, there is another issue to consider and that is what happens with SPAM due to the use of a Backup MX.
Your SME Server takes care of filtering a lot of SPAM by checking on the full username & domain at the time it is received.
For example if your server hosts example.com and someone sends mail to joeuser@example.com, the server will only accept the mail if joeuser is a local user/alias/group/pseudonym on the server. Otherwise, the mail is rejected during the SMTP transaction.
A backup mail server however, generally does not have a full list of users against which it can check if it should accept the mail for the given domain. Hence it will accept mail for invalid users.
So:
- If you trust the secondary MX, you will accept a lot of SPAM when the link comes up.
- If you don't trust it, you will cause a lot of SPAM backscatter as the mail has been accepted at the secondary MX and then later bounced by you.
- Stopping backscatter is why SME Server rejects invalid addresses during the initial SMTP transaction.
The SPAM backscatter can only be stopped if the secondary MX has a full list of users for your domain to allow filtering to occur.
But:
- You need to be able to configure this secondary MX with such user/domain lists
- You need to maintain these secondary configurations when users are added/deleted from your primary server configuration
- You need to test (regularly) if the secondary is successfully accepting/rejecting mail as required.
Quite a few sites have lost lots of mail through misconfigured backup MX servers. Unfortunately, the time when you find out they are misconfigured is when you go to use them, and then you find that the backup MX has changed configuration and bounced all of your mail.
Then you realise that this mail could have queued at the sender's site if there hadn't been a broken secondary MX bouncing the mail for you.
- If you bounce mail at your server, you have logs to show what's wrong.
- If your secondary MX bounces your mail, you usually have no way to determine what happened other than via reports from the original senders that your mail bounced.
Summary
In summary, if your server/Internet connection is available most (let's say >90%) of the time, you are generally better off without a secondary MX.
If your server/link is down more than this (e.g. dialup), you should not be delivering mail directly to your server.
If you still want to consider setting up a seconday MX, ensure that:
- you have fully control of the configuration of each of the email gateways for your domain
- each gateway can make decisions on whether to accept/reject mail for the users at the domain
Mail server on dynamic IP
Problems with running a mail server on SME server using a dynamic external IP from ISP
This information comes from http://bugs.contribs.org/show_bug.cgi?id=2057#c10
This is the chronological sequence of events that leads to issues with mail servers on dynamic IPs:
1) Server gets dynamic IP
2) Reboot/power fail (without updating dynamic DNS to "offline")
3) Another server/someone else is allocated your old IP while your server is down
4) The other server/person is running a mail server
5) The other server either gets your mail (which is bad) or bounces your mail (also bad)
You have no control over this issue and you will lose mail when it happens. If you have a dynamic IP, the recommended approach is to get someone with a static IP to queue your inbound mail and send it to you on a non-standard port, preferably with an authentication mechanism which queues the mail if the auth fails, just in case someone else happens to have a mail server on the same port (while highly unlikely, this is possible).
Whether this issue is really a problem to end users, depends on how much you "value" your mail. For a home user having their own mail server, it is probably not a great problem if some messages should happen to go astray, but for all other classes of users, you should really avoid running a mail server on a dynamic IP, without implementing a suitable queueing workaround as suggested. Some ISPs change the IP very infrequently eg yearly, so in those cases it is also not a significant problem. Many/most ISP's will issue a new IP every time a connection is lost & re-established, so these situations are more problematic.
How to re-apply procmail rules
If you have a folder of email that needs to have the procmail rules applied, then the trick is to be logged in as the email user, and then position your self in the home directory, and then this works:
su <username> -s /bin/bash cd ~ for m in <fullpath to maildirectory>/cur/*; do echo $m; procmail < $m && rm $m; done
Overview
The goal of the IPP2P project is to identify peer-to-peer (P2P) data in IP traffic.
It appears the ipp2p project may now be defunct (at February 2010), although the maintainer of the smeserver-ipp2p packages is still releasing updates for each SME server kernel upgrade (as at May 2012 for SME 7.6.x).
Installation
The rpms are currently located here
http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/
Please manually check the latest released rpm version numbers, and use those version numbers in the commands below.
To download do
mkdir -p /temp/ipp2p cd /temp/ipp2p wget http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/smeserver-ipp2p-1.0-2.el4.sme.noarch.rpm wget http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/ipp2p-0.8.2-4.el4.sme.i686.rpm wget http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/kmod-ipp2p-0.8.2-1.2.6.9_103.EL.i686.rpm wget http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/kmod-ipp2p-smp-0.8.2-1.2.6.9_103.EL.i686.rpm
If required, download and install the hugemem kmod module
wget http://jeremielorente.free.fr/linux/sme/contribs/ipp2p/kmod-ipp2p-hugemem-0.8.2-1.2.6.9_103.EL.i686.rpm
Install the rpms
cd /temp/ipp2p yum localinstall *.rpm
Configuration
Enabling
set configuration values for various networks
The default is disabled, ie no blocking of p2p
bit = bittorrent and ipp2p = are the common protocols, the others are less common and not as well tested.
config setprop ipp2p apple enabled config setprop ipp2p bit enabled config setprop ipp2p ares enabled config setprop ipp2p ipp2p enabled config setprop ipp2p soul enabled config setprop ipp2p winmx enabled
check settings are correct
config show ipp2p
apply changes and restart server
signal-event remoteaccess-update
Disabling
To disable blocking setprop to disabled
config setprop ipp2p apple disabled config setprop ipp2p bit disabled config setprop ipp2p ares disabled config setprop ipp2p ipp2p disabled config setprop ipp2p soul disabled config setprop ipp2p winmx disabled
apply changes and restart server
signal-event remoteaccess-update
Uninstallation
yum remove smeserver-ipp2p
Is this article helpful to you?
Please consider donating or volunteering
Thank you!
The server manager is the GUI front end for the firewall. The firewall is modified automatically in response to changes you make in the configuration, such as enabling/disabling services, marking them public/private, forwarding ports, etc.
If you wish to make changes beyond those provided for by the server manager, you can do so by setting DB records or providing custom templates. Only make these changes if you are sure you know what you are doing, incorrect settings will compromise security on your server.
FAQs
- I want to have two WAN addresses; one for the SMESERVER and another that needs to be treated like a "Local Network". I can't set any address from the WAN subnet as a "Local Network".
This is intended behaviour as SMESERVER is secure by design. If you need to do something like this, you should know what you are doing and understand what to poke under the covers.
DB Settings
- How do I allow public access to a service I've added to SME Server?
For this example the service you have installed is called 'manta' and 'nnn' is the TCP port number that needs to be opened. Watch your capitalization with the command below:
config set manta service access public status enabled TCPPort nnn
For UDP services, use UDPPort instead of TCPPort.
If you need to open multiple ports for one service you can use TCPPorts and UDPPorts. Port numbers are seperated with a comma, but without a space. Note that ranges of ports are defined with a : between the numbers in this case, instead of a -.
Note that you can also set restrictions with AllowHosts and DenyHosts:
config setprop manta AllowHosts 1.2.3.4,10.11.12.0/24 config setprop manta DenyHosts 16.17.18.18
Then, to activate, do:
signal-event remoteaccess-update
- I want to block traffic from some ip-addresses to my server on some port.
config setprop httpd-e-smith DenyHosts a.b.c.d,w.x.y.z signal-event post-upgrade signal-event reboot
Additional information on customizing iptables
Create a custom-named service definition in the configuration database. you can see the DB configuration
db configuration set <servicename> service
Apply your desired firewall restrictions to any existing SME 'service' or to a custom-named service that you have created. Combine a custom-named service with port-forwarding to create customized firewall rules.
db configuration setprop <servicename> TCPPort <portnumber> db configuration setprop <servicename> TCPPorts <portnumbers> db configuration setprop <servicename> UDPPort <portnumber> db configuration setprop <servicename> UDPPorts <portnumbers> db configuration setprop <servicename> status enabled|disabled db configuration setprop <servicename> access public|private db configuration setprop <servicename> AllowHosts a.b.c.d,x.y.z.0/24 db configuration setprop <servicename> DenyHosts e.f.g.h,l.m.n.0/24
Effectuate the changes you have made
signal-event remoteaccess-update
| Variable | Target | Default | Expected values |
|---|---|---|---|
| TCPPort | --proto tcp --dport <Ports> | Pre-configured for default services; no default for custom services | empty or a numerical or coma separated numbers |
| TCPPorts | --proto tcp --dports <Ports> | No default for custom services; Ranges of ports are defined with a : not a - | empty or a numerical or coma separated numbers |
| UDPPort | --proto udp --dport <Ports> | Pre-configured for default services; no default for custom services | empty or a numerical or coma separated numbers |
| UDPPorts | --proto udp --dports <Ports> | No default for custom services; Ranges of ports are defined with a : not a - | empty or a numerical or coma separated numbers |
| status | disabled | AllowHosts is set to "" (an empty string) unless the status is 'enabled' | 'enabled' or 'disabled' |
| access | private | AllowHosts is set to "" (an empty string) unless access is 'public' | 'private' for localhost and local network only (Server and gateway mode), 'public' for everywhere, 'localhost' for localhost only |
| AllowHosts | --src ..... --jump ACCEPT | Pre-configured for default services; no default for custom services. Default is '0.0.0.0/0' if service is enabled and public. | IP and netmask with this format 0.0.0.0/0, or coma separated list of these elements |
| DenyHosts | --src ..... --jump denylog | Pre-configured for default services; no default for custom services. If 'DenyHosts' is empty or does not exist then there are no '... --jump denylog' entries created in /etc/init.d/masq. | IP and netmask with this format 0.0.0.0/0, or coma separated list of these elements |
Custom templates
Block incoming IP address
- I want to block All traffic from some ip-addresses to my server.
Manual Method
Create a custom template and list the IP's
mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/ nano -w /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/40DenyRiffRaff
Now add the IP's you wish to block to the newly create file in the following format.
/sbin/iptables -A INPUT -s 69.212.12.76/32 -j DROP /sbin/iptables -A INPUT -s 88.28.215.11/32 -j DROP
expand and restart
/sbin/e-smith/expand-template /etc/rc.d/init.d/masq /etc/init.d/masq restart
To check the new block use the following command and look for the IP address you just DROP'ed. It should be listed in the "source" column.
iptables -L INPUT -v -n
Automated method
The above can be automated slightly.
First lets create a key where we can add IPs that we want to block:
config set ipblock configuration status enabled DenyHosts 208.100.26.0/24 logging disabled
As above, create the following template:
mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/ nano -w /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/40DenyRiffRaff
Paste this code:
{
use esmith::ConfigDB;
my $db = esmith::ConfigDB->open_ro
|| die 'Could not open configuration database';
# Completely block any riff raff
if ( ( my $status = $db->get_prop( 'ipblock', 'status' ) ) eq 'enabled' )
{
my $DenyHosts = $db->get_prop( 'ipblock', 'DenyHosts' ) || '';
if ( $DenyHosts ne '' ) {
my $logging = $db->get_prop( 'ipblock', 'logging' ) || 'disabled';
foreach my $host ( split( ',', $DenyHosts ) ) {
$OUT .= "\n";
$OUT .= "# Simple ipblock for riff raff\n\n";
if ( $logging eq 'enabled' ) {
$OUT .= "/sbin/iptables -A INPUT -s $host -j denylog\n";
}
else {
$OUT .= "/sbin/iptables -A INPUT -s $host -j DROP\n";
}
}
$OUT .= "\n";
}
else {
$OUT .= "# ipblock no DenyHosts set\n";
}
}
else {
$OUT .= "# ipblock disabled\n";
}
}
You can add multiple addresses separated by commas:
config setprop ipblock DenyHosts 208.100.26.0/24,1.2.3.4,5.6.0.0/16
You can disable this blocking with:
config setprop ipblock status disabled
If you want to log the dropped packets rather than just drop them:
config setprop ipblock logging enabled
Then expand and restart your firewall:
/sbin/e-smith/expand-template /etc/rc.d/init.d/masq /etc/init.d/masq restart
Block outgoing IPs or mac addresses
This section needs improvement.
See this forum post for clues re doing this, based in part on the concept of blocking incming traffic from specific external IPs.
http://forums.contribs.org/index.php/topic,46036.0/all.html
Formulation of suitable iptables rules will be required, use
man iptables
The template fragment needs to be placed in the right order, so that other rules do not negate the rule eg
20blockIP
Example: To block access based on the mac address of the NIC of the wokstation (not on IP)
mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/ pico -w /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/20Blockmac
Add the following code to the fragment and save
/sbin/iptables -A INPUT -m mac --mac-source XX:XX:XX:XX:XX:XX -j DROP
(Replace XX.XX.XX.XX.XX.XX with actual mac address)
expand-template /etc/rc.d/init.d/masq /etc/init.d/masq restart
Check that blocking works as expected
To see the iptables that are in effect on your server, issue the command
iptables --list
or
iptables -L
Block outgoing ports
- I want to block outgoing traffic from my server.
These commands are based on http://bugs.contribs.org/show_bug.cgi?id=2977
Please check for the latest attachments (custom template fragments) to this bug.
At present, traffic is only blocked if it originates on the primary local network. No processing is performed on traffic addressed to the LAN IP, WAN IP or loopback address of the SME.
Download custom templates and configure ports with db command
mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq cd /etc/e-smith/templates-custom/etc/rc.d/init.d/masq wget -O 91adjustPortBlocks http://bugs.contribs.org/attachment.cgi?id=1395 wget -O 42SetupPortBlocks http://bugs.contribs.org/attachment.cgi?id=1389
Create desired db entries to suit the ports & protocols you want to block
config setprop masq TCPBlocks address:port config setprop masq UDPBlocks address:port
eg to block all outbound traffic except that passed by the smtp & httpd proxies
config setprop masq TCPBlocks 0.0.0.0/0:1-65535 config setprop masq UDPBlocks 0.0.0.0/0:1-65535
eg to leave open some ports ie 222 & 2000-2010, block in ranges
config setprop masq TCPBlocks 0.0.0.0/0:1-221,0.0.0.0/0:223-1999,0.0.0.0/0:2011-65535
Update the config changes and restart masq
signal-event remoteaccess-update /etc/init.d/masq restart
Bypass Proxy
- You have Transparent Proxy enabled (the default) but want to allow this to be selectively bypassed.
These commands are based on http://bugs.contribs.org/show_bug.cgi?id=2374
Please check for the latest attachments (custom template fragments) to this bug.
Download custom templates and configure ports with db command
mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq cd /etc/e-smith/templates-custom/etc/rc.d/init.d/masq wget -O 35transproxy http://bugs.contribs.org/attachment.cgi?id=1410 wget -O 90adjustTransProxy http://bugs.contribs.org/attachment.cgi?id=2178
Create desired db entries for the clients or sites you want to allow
config setprop squid BypassProxyTo 162.23.23.125 config setprop squid BypassProxyFrom a.b.c.d,x.y.z.0/0 expand-template /etc/rc.d/init.d/masq /etc/init.d/masq restart
If the setting changes do not appear to take effect, do the following
signal-event reboot
To add a BypassProxyFrom IP & retain existing IPs without re-entering them, do the following
config setprop squid BypassProxyFrom a.b.c.d,$(config getprop squid BypassProxyFrom) expand-template /etc/rc.d/init.d/masq /etc/init.d/masq restart
Followed if necessary by
signal-event reboot
To remove a specific entry but leave other existing entries unchanged
config setprop squid BypassProxyFrom \ $(config getprop squid BypassProxyFrom | \ sed -e 's/entry-to-be-removed//' -e 's/^,//' -e 's/,$//' -e 's/,,//')
where entry-to-be-removed is the IP to be removed
Note: The first sed is to remove the entry, the last second is to remove the comma at the beginning, the second for a comma at the end and the last to remove the double comma when an entry is removed at the middle of the list.
Disable bypass:
config delprop squid BypassProxyFrom config delprop squid BypassProxyTo expand-template /etc/rc.d/init.d/masq service masq restart signal-event reboot
Open Ports in Private Server/Gateway Mode
- I want to hide all ports, so I put my SMESERVER in PRIVATE SERVER/GATEWAY mode. I can still see some ports are open.
Certain services are still open on the WAN interface in PRIVATE SERVER/GATEWAY mode. Those services can be set to absolute private from the command line by:
config setprop masq Stealth yes config setprop ftp access private config setprop smtpd access private config setprop dnscache access private config setprop httpd-e-smith access private config setprop oidentd access private config setprop modSSL access private config setprop ssmtpd access private config setprop sshd access private config setprop imaps access private config setprop ldap access private config setprop pop3 access private config setprop pop3s access private config setprop nmbd access private config setprop smbd access private signal-event post-upgrade signal-event reboot
iBays
To enable a Recycle Bin for an iBay
db accounts setprop <ibayname> RecycleBin enabled db accounts setprop <ibayname> KeepVersions enabled signal-event ibay-modify <ibayname>
LDAP
In the unlikely event your LDAP directory is corrupted you can do the following to "rebuild" your directory
sv d /service/ldap mv /home/e-smith/db/ldap/domain.xxx.ldif /home/e-smith/db/ldap/domain.xxx.backup find /var/lib/ldap -type f | xargs rm -f expand-template /home/e-smith/db/ldap/ldif sv u /service/ldap
Console Login
To change the console from automatic login to being prompted to login
config set ConsoleMode login signal-event console-save
General Hardware Information
Hard Drives
A software RAID array will be automatically configured as part of the installation process for servers which contain multiple hard drives. This is to ensure redundancy, so if one disk fails the system will still function.
The specifics of the RAID setup depends on the number of drives available, to balance redundancy and capacity.
The root and swap volumes are configured using LVM on the RAID device /dev/md1 as follows:
- 1 drive - no RAID
- 2 drives - RAID 1
- 3 drives - RAID 1 + hot spare
- 4 drives - RAID 6
- 5+ drives - RAID 6 + hot spare
The /boot volume and EFI partition if necessary is always a non-LVM RAID 1 array on the device /dev/md0.
If you use a hardware RAID controller to manage your drives, this should be configured to present a single volume, which SME will configure without software RAID.
Default RAID Rationale
The differences in RAID layout between SME Server 10 and previous versions is summarised below:
| Number of Drives | SME Server 10 | Previous Versions |
|---|---|---|
| 1 | No software RAID | Degraded RAID 1 |
| 2 | Software RAID 1 | |
| 3 | RAID 1 + hot spare | |
| 4 | RAID 6 | RAID 5 + hot spare |
| 5 | RAID 6 + hot spare | |
| 6 | ||
| 7+ | RAID 6 + hot spare | |
The main differences are no degraded RAID 1 for a single disk install, which better supports virtualised and hardware RAID use cases, and a preference for RAID 6 over RAID 5. This is to reduce the risk of a single disk failure bringing down the array. While consumer hard drives have got significantly larger over time, their unrecoverable read error rate (URE) has remained at 1 per 10^14 bits, or 12TB. As an example, imagine a server with 5 x 4TB drives. Under previous versions of SME Server this would have been configured as a 4 disk RAID 5 array with 1 hot spare. If one drive failed, the hot spare would become active and the array would begin to rebuild. This would require reading all 3 disks and, at some point during that 12TB operation, it’s very likely that an unrecoverable error would be encountered. At this point, the whole array would fail. In comparison, a RAID 6 array is tolerant to two disk failures. While this does not entirely solve the risk of a URE during rebuild, it significantly reduces the likelihood of it taking down the array. Note: RAID is a convenient method of protecting server availability from a drive failure. It does not remove the need for regular backups, which can be configured using the Server Manager.
Disk Layout
Mirroring drives in the same IDE channel (eg. hda and hdb) is not desirable. If that channel goes out, you may lose both drives. Also, performance will suffer slightly.
The preferred method is to use the primary location on each IDE channel (eg. hda and hdc). This will ensure that if you lose one channel, the other will still operate. It will also give you the best performance.
In a 2 drive setup put each drive on a different IDE channel:
IDE 1 Primary - Drive 1
IDE 1 Secondary - CDROM
IDE 2 Primary - Drive 2
Obviously this section is obsolete with SATA hard drives because each disk has its own channel.
Identifying Hard Drives
It may not always be obvious which physical hard drive maps to which logical device. The first step would be to be able to identify all block devices present on your server. This could be done by using two commands
lsblk
or the following
findmnt
Then, once you identified a block device , the simplest method to verify which physical drive it is, is using S.M.A.R.T. capability to map the serial number on the physical package with that displayed by smartctl. Assuming the device of interest is sda, (a SCSI drive), then you would issue the following command as root:
smartctl -i /dev/sda
Or if an IDE Drive
smartctl -i /dev/hda
Adding Additional Drives
For servers which were installed with 2+ drives and have a working RAID array, it is possible to add an additional drive which will become a hot spare, ready to be activated in case of drive failure.
See also AddExtraHardDisk. It's an alternative for part of the data if you have only one drive and you want to use RAID1. But better solution is to reinstall SME10 with 2 drives.
Ensure that any new drives are the same size or larger than your existing drives.
- Shut down the machine
- Install one additional drive at a time
- Boot up
- At the login prompt log on as admin with the root password to get to the admin console
- Go to #5 Manage disk redundancy
- Accept the option to add an additional drive
If the Manage disk redundancy page displays the message "The free disk count must equal one" and "Manual intervention may be required", then you probably have additional hard drives that need to be disconnected while the RAID is set up. An external USB drive will have this effect, and should be unplugged.
Reusing Hard Drives
- MBR formatted disks
If it was ever installed on a Windows machine, or any of the *BSDs, (or in some cases an old system with RAID and/or LVM) then you will need to clear the MBR first before installing it.
From the linux command prompt, type the following:
#dd if=/dev/zero of=/dev/hdx bs=512 count=1
or
#dd if=/dev/zero of=/dev/sdx bs=512 count=1
You MUST reboot so that the empty partition table gets read correctly.
For more information, check: http://bugs.contribs.org/show_bug.cgi?id=2154
- For disks previously formatted as GPT this is insufficient. It's probably best to use gdisk or parted or partx to delete the partitions; there are other tools that will work. Parted has limited support for LVM.
- To remove the (hardware) RAID configuration that is stored at the end of the drive, do:
#dd if=/dev/zero of=/dev/sdx bs=512 count=2048 seek=$((`blockdev --getsz /dev/sdx` - 2048))
Upgrading the Hard Drive Size
Note: these instructions are only applicable if you have a RAID system with more than one drive. They are not applicable to a single-drive RAID 1 system, and increasing the useable space on such a system by cloning the existing single drive to a larger drive is not supported. See http://bugs.contribs.org/show_bug.cgi?id=5311
- CAUTION MAKE A FULL BACKUP!
- Ensure you have e-smith-base-4.16.0-33 or newer installed. [or Update to at least 7.1.3]
HD Scenario - Current 250gb drives, new larger 500gb drives
- Shut down and install one larger drive in system for one old HD. Unplug any USB-connected drives.
- Boot up and login to the admin console and use option 5 to add the new (larger) drive to system.
- Wait for raid to fully sync.
- Repeat steps 1-3 until all drives in system are upgraded to larger capacity.
- Ensure all drives have been replace with larger drives and array is in sync and redundant!
- Issue the following commands:
mdadm --grow /dev/md2 --size=max pvresize /dev/md2 lvresize -l +100%FREE main/root ext2online -C0 /dev/main/root
In the last command above, the -C0 is: dash C zero
If you receive an "command not found" error, try this:
resize2fs /dev/mapper/main-root &
TIP: I put an "&" at end to allow it to run in background even if I close ssh session.
Notes :
- All of this can be done while the server is up and running with the exception of #1.
- These instructions should work for any raid level you have as long as you have >= 2 drives
- If you have disabled lvm , you don't need the pvresize or lvresize command, therefore the final line becomes
ext2online -C0 /dev/md2 #(or whatever / is mounted to)
or If you receive an "command not found" error, try this:
resize2fs /dev/md2 &
Replacing and Upgrading a Hard Drive after HD fail
Note: See Bugzilla: 6632 and Bugzilla:6630 a suggested sequence for Upgrading a Hard Drive size is detailed after issue when attempting to sync a new drive when added first as sda.
Note: These instructions are applicable if you have a faulty HD on a RAID system with more than one drive and intend to upgrade the sizes as well as replacing the failed HD. They are not applicable to a single-drive RAID 1 system, and increasing the useable space on such a system by cloning the existing single drive to a larger drive is not supported. See http://bugs.contribs.org/show_bug.cgi?id=5311
- CAUTION MAKE A FULL BACKUP!
- Ensure you have e-smith-base-4.16.0-33 or newer installed. [or Update to at least 7.1.3]
HD Scenario - Current 250gb drives, new larger 500gb drives
- Remove failed HDD from system, ensure remaining drive is on sda on its own and boot up.
- Shutdown, connect one new 500gb drive as sdb and boot up
- Login to the admin panel and manage raid to add new (larger) drive to system.
- Wait for raid to fully sync
- Do full reboot with those 2 drives in place (1 original, 1 new)
- Shutdown again, disconnect the original drive, and connect the new drive just sync'd as sda (in place of original)
- Boot up again with just the one new drive in place, and confirm it boots OK.
- Shutdown, and connected the other 500gb drive as sdb
- Boot up login to admin panel and add sdb to the array, and wait for raid to fully sync.
- Reboot and ensure all drives have been replaced with larger drives and array is in sync and redundant!
- Issue the following commands:
mdadm --grow /dev/md2 --size=max pvresize /dev/md2 lvresize -l +100%FREE main/root ext2online -C0 /dev/main/root
In the last command above, the -C0 is: dash C zero
If you receive an "command not found" error, try this:
resize2fs /dev/mapper/main-root &
TIP: I put an "&" at end to allow it to run in background even if I close ssh session.
Notes :
- These instructions should work for any raid level you have as long as you have >= 2 drives
- If you have disabled lvm , you don't need the pvresize or lvresize command, therefore the final line becomes
ext2online -C0 /dev/md2 #(or whatever / is mounted to)
or If you receive an "command not found" error, try this:
resize2fs /dev/md2 &
RAID Notes
Many on-board hardware raid cards are in fact software RAID. Turn it off as cheap "fakeraid" cards aren't good for Linux. You will generally get better performance and reliability with Linux Software RAID (http://linux-ata.org/faq-sata-raid.html). Linux software RAID is fast and robust.
If you are insistent on getting a hardware RAID, buy a well supported RAID card which has a proper RAID BIOS. This hides the disks and presents a single disk to Linux (http://linuxmafia.com/faq/Hardware/sata.html). Please check that it is supported by the kernel and has some form of management. Also avoid anything which requires a driver. Try googling for the exact model of RAID controller before buying it. Please note that you won't get a real hardware raid controller cheap.
It rarely happens, but sometimes when a device has finished rebuilding, its state doesn't change from "dirty" to "clean" until a reboot occurs. This is cosmetic
Periodic scrub of RAID arrays
A Periodic scrub of RAID arrays (weekly raid check) is performed every week on Sunday at 04:22 local time, refer Bugzilla:3535 and Bugzilla:6160 for more information.
Theses operations are logged, however, no emails will be sent to admin as of the release of packages associated with Bug #6160 or the release of the 8.1 ISO.
Receive periodic check of RAID by email
There are routines in SME Server to check the RAID and sent mail to the admin user, when the RAID is degraded or when the RAID is resynchronizing. But the admin user may receive a lot of emails and sometimes messages can be forgotten. So the purpose is to have a routine which sends email to the user of your choice each week.
nano /etc/cron.weekly/raid-status.sh
You have to change the variable DEST="stephane@your-domaine-name.org" to the email you decide to use.
#!/bin/sh # cron.weekly/mdadm-status -- weekly status of the RAID # 2013 Pierre-Alain Bandinelli # distributed under the terms of the Artistic Licence 2.0 # Get status from the RAID array and send the details by email. # Email will go to the address specified in the commandline. set -eu MDADM=/sbin/mdadm [ -x $MDADM ] || exit 0 # package may be removed but not purged DEST="stephane@your-domaine-name.org" exec $MDADM --detail $(ls /dev/md*) | mail -s "RAID status SME Server" $DEST
save by ctrl+x
chmod +x /etc/cron.weekly/raid-status.sh
each sunday a 4h00 AM you will receive a mail which looks to this :
/dev/md1:
Version : 0.90
Creation Time : Sun Jan 6 20:50:41 2013
Raid Level : raid1
Array Size : 104320 (101.89 MiB 106.82 MB)
Used Dev Size : 104320 (101.89 MiB 106.82 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent
Update Time : Sun Dec 22 04:22:42 2013
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 28745adb:d9cff1f4:fcb31dd8:ff24cb0c
Events : 0.208
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
/dev/md2:
Version : 0.90
Creation Time : Sun Jan 6 20:50:42 2013
Raid Level : raid1
Array Size : 262036096 (249.90 GiB 268.32 GB)
Used Dev Size : 262036096 (249.90 GiB 268.32 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Sun Dec 22 05:30:36 2013
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : c343c79e:91c01009:fcde78b4:bad0b497
Events : 0.224
Number Major Minor RaidDevice State
0 8 2 0 active sync /dev/sda2
1 8 18 1 active sync /dev/sdb2
If you want to test the message sent without waiting the next sunday, you can do
/etc/cron.weekly/raid-status.sh
nospare
If you use the commandline parameter nospare during installation ("sme nospare"), the system will still count the missing spare towards the number of drives. A system with 6 physically present harddrives thus will be formated Raid6 _not_ Raid5. Resulting capacity of course will be "n-2". Note: with the release of version 7.6 and 8.0, the commandline parameter "sme nospare" has been changed to "sme spares=0" . In addition, you may also select the number of spare(s) implemented [0,1,or 2].
remove the degraded RAID message
When you install the smeserver with one drive with a degraded raid, you will see a 'U_' state but without warnings. If you want to leave just one 'U' in the /proc/mdstat and stop all future questions about your degraded raid state, then :
mdadm --grow /dev/md0 --force --raid-devices=1 mdadm --grow /dev/md1 --force --raid-devices=1
after that you will see this
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0]
255936 blocks super 1.0 [1/1] [U]
md1 : active raid1 sda2[0]
268047168 blocks super 1.1 [1/1] [U]
bitmap: 2/2 pages [8KB], 65536KB chunk
unused devices: <none>
Resynchronising a Failed RAID
Sometimes a partition will be taken offline automatically. Admin will receive an email DegradedArray event on /dev/md2.
smartctl -a /dev/hda
Where hda is the device to be checked; check all of them.
You may check the health of your array using the Admin Console. Login as root, type console. Select Item 5. "Manage disk reduncancy"
--------Disk Reduncancy status as of Thursday Dec 22 -------
Current RAID status:
Personalities : [raid1]
md2 : active raid1 hda2[0] <-- NOTICE hdb2[#] is missing. Means hdb2[#] failed.
38973568 blocks [2/1] [U_]
md1 : active raid1 hda1[0] hdb1[1]
104320 blocks [2/2] [UU]
unused devices: <none>
Only Some of the RAID devices are unclean. <-- NOTICE This message and
Manual intervention may be required. <-- this message.
Notice the last 2 sentences of the window above. You have some problems.
If your system is healthy however the message you will see at the bottom of Raid Console window is:
All RAID devices are in clean state
If you have no software RAID devices you will see the message at the bottom of the Console window:
Your system only has a single disk drive installed or is using hardware
mirroring. If you would like to enable software mirroring, please shut
down, install a second disk drive (of the same capacity) and then return
to this screen.
Additionally, the details of the raid can be seen by inspecting the mdstat file from the shell prompt.
[root@sme]# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 hda3[0] hdb3[1]
38837056 blocks [2/2] [UU]
md2 : active raid1 hdb2[1] <-- Shows current active partition - note there is one missing
1048704 blocks [2/1] [_U] <-- '_' = partition missing from array
md0 : active raid1 hda1[0] hdb1[1]
255936 blocks [2/2] [UU]
Make a note of the raid partition that has failed, shown by [_U]
In this case it is md2, the device being /dev/md2.
The failed drive partition is indicated by the '_' underline character. In the above example _U indicates that the first drive partition on md2, (Multi-Device 2) has failed. The second drive partition on md2, symbolized by the character 'U' is still part of the md2. If the second drive partition had failed, that is hdb2 then the details would be reversed. E.g. [U_] . Placing the _ Underline second in the details.
Determine the missing physical partition, Look carefully at the sample above and fill in the gap for which drive is missing.
In this example, it's hda2, the device being /dev/hda2
md1 : active raid1 hda3[0] hdb3[1] md2 : active raid1 hda2[0] hdb2[1] <--- In the above sample hda2[0] is missing md0 : active raid1 hda1[0] hdb1[1]
If the raid has a failed disk that has not yet been kicked out of the array then mdstat will show something like the following:
md2 : active raid1 hda2[0](F) hdb2[1] <-- Shows current active partition - with one FAILED (F)
1048704 blocks [2/1] [_U] <-- '_' = partition missing from array
In this case before you add the disk back in you will need to remove the disk as per:
[root@sme]# mdadm --remove /dev/md2 /dev/hda2
However if the drive has already been removed by the operating system then removing the drive is unnecessary. To determine this use the command:
mdadm --query --detail /dev/md2
Of course use the proper md# based on your configuration. This command will give you several lines of data, including the size of the array. Near the end of the output you will see the following if the drive has been removed already. There is no need to remove the drive since it has already been removed.
Number Major Minor RaidDevice State
0 3 2 0 active sync /dev/hda2
1 0 0 - removed <-- NOTE THIS
To add the physical partition back and rebuild the raid partition.
[root@sme]# mdadm --add /dev/md2 /dev/hda2
Once you type the command the following message will appear, appropriate for your device.
[root@sme] mdadm: hot added /dev/hda2
It important to know that your devices are likely to be different, E.G your device could be /dev/sda2 or you may have more than two disks, including a hot standby. These details can always be determined from the mdstat file. Once the raid resync has been started, the progress will be noted in mdstat. You can see this real time by:
[root@sme]# watch -n .1 cat /proc/mdstat
or you can see this in a snapshot by:
[root@sme]# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 hda3[0] hdb3[1]
38837056 blocks [2/2] [UU]
md2 : active raid1 hda2[2] hdb2[1]
1048704 blocks [2/1] [_U]
[=>...................] recovery = 6.4% (67712/1048704) finish=1.2min speed=13542K/sec
md0 : active raid1 hda1[0] hdb1[1]
255936 blocks [2/2] [UU]
When recovery is complete, the partitions will all be up:
[root@sme]# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 hda3[0] hdb3[1]
38837056 blocks [2/2] [UU]
md2 : active raid1 hda2[0] hdb2[1]
1048704 blocks [2/2] [UU]
md0 : active raid1 hda1[0] hdb1[1]
255936 blocks [2/2] [UU]
If this action is required regularly, you should test your disks for SMART errors and physical errors, check your disk cables, and make sure no two hard drives share the same IDE port. See also: http://wiki.contribs.org/Monitor_Disk_Health
Also check your driver cards, since a faulty card can destroy the data on a full RAID set as easily as it can a single disk.
Convert Software RAID1 to RAID5
- Login as root
- Move to /boot (we must create a new initrd image to load raid5 driver). cd /boot
- Make a backup copy mv initrd-`uname -r`.img initrd-`uname -r`.img.old
- Create the new image mkinitrd --preload raid5 initrd-`uname -r`.img `uname -r`
- Shut down and install new drive(s) in system.
- Boot up with SME cd and enter the rescue mode. sme rescue
- Skip network setup.
- Skip mounting the current SME installation.
- Now, create on the new drive(s) the correct partition table. sfdisk -d /dev/sda > tmp.out sfdisk /dev/sdc < tmp.out
- Repeat the last step for each new hd (sdd, sde etc.).
- Create the new array mdadm --create /dev/md2 -c 256 --level=5 --raid-devices=2 /dev/sda2 /dev/sdb2 mdadm: /dev/sda2 appears to be part of a raid array: level=raid1 devices=2 ctime=Fri Dec 18 13:17:49 2009 mdadm: /dev/sdb2 appears to be part of a raid array: level=raid1 devices=2 ctime=Fri Dec 18 13:17:49 2009 Continue creating array? y mdadm: array /dev/md2 started.
- Wait for resync; monitor the status with cat /proc/mdstat root# cat /proc/mdstat Personalities : [raid0] [raid1] [raid5] md2 : active raid5 sdb1[2] sda1[0] 1048512 blocks level 5, 256k chunk, algorithm 2 [2/1] [U_] [==>..................] recovery = 12.5% (132096/1048512) finish=0.8min speed=18870K/sec
- Reboot exit
- Login as root
- Add the new drives to the array mdadm --add /dev/md2 /dev/sdc2
- Repeat the last step for each new hd (sdd2, sde2 etc.)
- Grow the array mdadm --grow /dev/md2 --raid-devices=N
- N is the total number of drives: minimum is 3
- Wait for array reshaping. This part can take a substantial amount of time; monitor it with cat /proc/mdstat root# cat /proc/mdstat Personalities : [raid0] [raid1] [raid5] md2 : active raid5 sdc1[2] sdb1[1] sda1[0] 1048512 blocks super 0.91 level 5, 256k chunk, algorithm 2 [3/3] [UUU] [==>..................] reshape = 12.5% (131520/1048512) finish=2.5min speed=5978K/sec
- Issue the following commands: pvresize /dev/md2 lvresize -l +100%FREE main/root resize2fs /dev/main/root
Notes :
- If you have disabled lvm
- you don't need the pvresize or lvresize command
- the final line becomes resize2fs /dev/md2 (or whatever / is mounted to)
- More info: http://www.arkf.net/blog/?p=47
Add another Raid to mount to /home/e-smith/files
this is inspired from previous content of AddExtraHardDisk and particularly the part AddExtraHardDisk#Additional steps to create a raid array from multiple disks but updated to 2022 and SME10
1 you need to check what disk you want, using lsblk
# lsblk --fs
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
├─sda1 vfat B93A-85A4 /boot/efi
├─sda2 xfs 89e9cc9e-d3d2-4d02-bad5-2698aea0a510 /boot
├─sda3 swap 64d21f89-4d7c-417a-907e-34236f6cd0be [SWAP]
└─sda4 xfs 65bf712c-2186-4524-aae8-edd8151de1e7 /
sdb
sdc
then you can create the Raid array. We assume you only need onethen you need to rebuild the grub.conf, depending on your system is EFI or legacy use the appropriate command#EFI
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg #Legacy grub2-mkconfig -o /boot/grub2/grub.cfg
Raid partition, and hence do not need to partition it.
#create array
mdadm --create --verbose /dev/md11 --level=1 --raid-devices=2 /dev/sdb /dev/sdc
# add to mdadm.conf
mdadm --detail --scan --verbose /dev/md11 >> /etc/mdadm.conf
then format it and enable quotas. If you want to add a LVM Layer, this is just before that !
mkfs.xfs /dev/md11
now you have
# lsblk --fs
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
├─sda1 vfat B93A-85A4 /boot/efi
├─sda2 xfs 89e9cc9e-d3d2-4d02-bad5-2698aea0a510 /boot
├─sda3 swap 64d21f89-4d7c-417a-907e-34236f6cd0be [SWAP]
└─sda4 xfs 65bf712c-2186-4524-aae8-edd8151de1e7 /
sdb linux_raid_member localhost.localdomain:11 6c4b9640-7349-15fe-28b1-78843d9a149a
└─md11 xfs 0ab4fe2a-aa81-4728-90d8-2f96d4624af8
sdc linux_raid_member localhost.localdomain:11 6c4b9640-7349-15fe-28b1-78843d9a149a
└─md11 xfs 0ab4fe2a-aa81-4728-90d8-2f96d4624af8
then you need to mount it temporary to move your content
mkdir /mnt/newdisk
mount /dev/md11 /mnt/newdisk
rsync -arv /home/e-smith/files/ /mnt/newdisk
When happy with result simply add an entry to you fstab, according to last lsblk output in this case you should add
UUID=0ab4fe2a-aa81-4728-90d8-2f96d4624af8 /home/e-smith/files xfs uquota,gquota 0 0
To have the disk mounted on reboot, you need to alter grub
vim /etc/default/grub
and alter the command line to add either "rd.md=1 rd.md.conf=1 rd.auto=1" or specifically add the uuid to mount (obviously if you add a LVM layer you will rather need to add something like rd.lvm.lv=mylvm/video rd.lvm.lv=mylvm/files)
1GRUB_TIMEOUT=5
2GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
3GRUB_DEFAULT=saved
4GRUB_DISABLE_SUBMENU=true
5GRUB_TERMINAL_OUTPUT="gfxterm"
6GRUB_CMDLINE_LINUX="rhgb quiet rootflags=uquota,pquota rd.md=1 rd.md.conf=1 rd.auto=1"
7GRUB_DISABLE_RECOVERY="false"
8GRUB_BACKGROUND="/boot/grub2/smeserver10.png"
9GRUB_GFXMODE="1024x768"
10GRUB_THEME="/boot/grub2/themes/koozali/theme.txt"
then you need to make sure dracut will add the drivers
vim /etc/dracut.conf
and alter the line needed (you probably will need to uncomment this line and add mdraid between the quote)
19# dracut modules to add to the default
20add_dracutmodules+="lvm mdraid"
21
22# install local /etc/mdadm.conf
23mdadmconf="yes"
24
25# install local /etc/lvm/lvm.conf
26lvmconf="yes"
Finally rebuild the initfs
cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --add="lvm mdraid" /boot/initramfs-$(uname -r).img $(uname -r) --force
Copy data from one disk of an old Raid mirror disk
Let's say you want to copy the huge amount of data you excluded from the backup to migrate from SME9 to SME10 and now you want to copy this to your new server.
This How-To assume your current install is without LVM. An extra trick is needed if you have a LVM and previous SME9 also. You simply need to rename the vg group either of the old SME or new one using a rescue disk or another Linux distro, see UpgradeDisk#Moving from SME 8.x to SME 9.x.
- put one of the old drives in the server or in an external case and connect it
- use lsblk to identify the drive
- adapt the following commands
# lsblk
sdd 8:48 0 931,5G 0 disk
├─sdd1 8:49 0 250M 0 part
└─sdd2 8:50 0 931,3G 0 part
We assume that sd1 was the boot partition and the stuff we want is in sdd2
#assemble and run on degraded
mdadm -A /dev/md126 /dev/sdd2 --run
now let's try to mount, it will work only if you had no LVM, or it will return this
# mkdir /mnt/olddisk/
# mount /dev/md126 /mnt/olddisk/
mount: filesystem « LVM2_member » unknown
you can skip this step if you did not got the LVM error. Then we need to activate the LVM, and we can assume you might need also to install lvm stuffs...
# yum install lvm2 -y
vgchange -a y main
mount /dev/mapper/main-root /mnt/olddisk/
It is now time to copy your stuffs.
rsync -arvHAX /mnt/olddisk/home/e-smith/files/ /home/e-smith/files
then to remove safely your disk
umount /dev/mapper/main-root
vgchange -a n main
mdadm --stop /dev/md126
Removable Media
CD or DVD Drives are likely to be located at /media/cdrecorder
to read from a cd, first mount with
mount /media/cdrecorder
or check with
cat /etc/fstab |grep media
USB Drives
Releasing Contribs
Writing
The Developers Manual and page Web Application RPM have information required to write a contrib for SME Server
Releasing
For consistency between contribs it's suggested that you follow these guidelines
- Announce your new rpm on the forum, make the subject similar to below
[ANNOUNCE] rpm name
- Submit a bug to the bugtracker, for SME Contribs, asking for a new component to be added with your rpm name. Include links to download your rpm including your source RPM. Include a short description, perhaps the description from the spec file.
- Your contrib should preferably be in the SME Contribs repository.
SME Contribs Repositories
The SME Contribs repository contain RPMs that stored on the sourceforge smecontribs repository and have passed some level of verification from senior SME developers for their initial importation.
To get your package in the SME Contribs repository follow the instructions at Package Modification.
After initial verification for importation, bug fix and verification is the sole responsibility of the contributor or the person who step in to fix the bug and or built the package. He may ask help from somebody with the appropriate configuration, hardware to assist him doing so if he feels himself unable to test it. Moving the resulting rpm from smebuild or smetest to smecontribs is the responsibility of the person who built the package.
SME Addon Repositories
The SME Addon repository will contain RPMs that have been through a review process, RPMs in the SME Addon repository will be able to be installed directly from the Software Installer panel in the server-manager. Currently the SME Addon repository does not hold any contribs, but this will change in the future.
The process for having your contrib included in SME Addon is currently being outlined and developed and therefore is far from complete. It begins by adding your rpm to SME Contribs.
Uploading contribs to contribs.org
If you have created a contrib and wish to share with everyone then the best method is to add it to SME Contribs.
Alternatively you can upload to your mirror.contribs.org/smeserver/contribs directory. To do so you can use ssh, rysnc, sftp or scp to upload to your contrib directory.
It is recommend that you setup public key access for your account. The following articles may be of help to you:
Using SSH
You can log in using a shell account with limited access, to do this you can either use your password or set up ssh (see note about permissions of ssh keys) keys:
ssh username@shell.contribs.org
You will then see either ssh authenticated key login:
[localuser@localhost ~]$ ssh username@shell.contribs.org Enter passphrase for key '/home/username/.ssh/id_rsa': Last login: Tue Oct 23 15:09:50 2007 from xxx.xxx.xxx.xxx [username@shell ~]$
or if using a shell password:
[localuser@localhost ~]$ ssh username@shell.contribs.org username@shell.contribs.org's password: Last login: Tue Oct 23 15:09:50 2007 from xxx.xxx.xxx.xxx [username@shell ~]$
There is limited shell access as mentioned and the available commands are:
cat bash cat chgrp chmod chown clear cp dircolors echo egrep find grep hostname id less ln ls mkdir more mv newgrp passwd rm rmdir rsync scp sftp sed sort touch uniq vi xargs
You can also change your password within your shell account by typing:
passwd
Now upload your contribs and share with everyone ;)
Using RSYNC
Make sure you have a working copy of your contrib directory, for example to use rsync we will grab a copy by typing...
rsync -avP username@shell.contribs.org:~username/ /tmp/test/
Now make your /tmp/test directory looks like what you wish to see on your mirror.contribs.org/smeserver/contribs directory, if you wish to remove a contrib from your directory you can do so by typing the following...
rsync -avP --delete /tmp/test/ username@shell.contribs.org:~username/
And that's it now wait for mirrors to sync and have a beer.
Using SFTP
Log in using...
sftp username@shell.contribs.org
Now you will be at the...
sftp>
Now you can create your directory ready for uploading contribs...
sftp> mkdir smecontribs sftp>
Then upload your local copy of your new completed contrib to your contrib directory by typing...
sftp> cd smecontribs sftp>
Show we are in the right directory where we want the contrib to show...
sftp> pwd Remote working directory: /home/byte/smecontribs sftp>
Make sure we have our local server directory set correctly by typing...
sftp> lls anaconda-ks.cfg Desktop install.log install.log.syslog sftp>
Above show's we are not so to move in to our local working directory type...
sftp> lcd /tmp/sftptest sftp> lls smeserver-contrib-1.rpm smeserver-contrib.rpm sftp>
We are ready to upload our 2 new contribs by typing...
sftp> put smeserver-* Uploading smeserver-contrib-1.rpm to /home/username/smecontribs/smeserver-contrib-1.rpm smeserver-contrib-1.rpm 100% 0 0.0KB/s 00:00 Uploading smeserver-contrib.rpm to /home/username/smecontribs/smeserver-contrib.rpm smeserver-contrib.rpm 100% 0 0.0KB/s 00:00 sftp>
And thats it wait for mirrors to sync and sit back/relax.
Using SCP
You can send files by using scp:
[local@localhost contribs]$ scp username@shell.contribs.org:~username/smeservercontribs/RPMS Enter passphrase for key '/home/username/.ssh/id_rsa': smeserver-contrib 100% 103 0.1KB/s 00:00 [local@localhost contribs]$
Sit back and wait for mirrors to sync.
Bug Tracker
Please help us to make our software as reliable as possible. Please report possible bugs only via the bug tracker and encourage others to do the same.
Bugzilla is just a standard communication format for problem solving. Replies may seem abrupt but that is just the nature of the medium. Don't be concerned by a stark Wontfix or Invalid, it's how the system works. Discussions are for forums
Find, report and fix bugs here http://bugs.contribs.org
Bugzilla is easy
A frequent complaint is Bugzilla is too hard or convoluted, but it is hard to justify the complaint.
- You open an account
- Then report your problem
SME is run by volunteers so we expect you to help by searching for existing bugs and by filing your bug in the format described below. To begin you need only read 'searching' and 'reporting'.
Searching Bugs
Have you searched bugzilla, and read the SME Server Manual and FAQ ?
Simple and Advanced aren't necessarily easy so we added prepared searches to the bugzilla page headers.
- Simple
Select a Status and a Product category and enter your search term
- Advanced
You can restrict searches by using a selection or multiple selections of a category
Use Control Click to select/deselect from selection sets, if none are selected the defailt is all.
- Reports
A wiki page with more prepared searches.
- Matrix
A Grid with one view of current bugs, more are available in Reports.
- Recent
Bugs with activity in the last two days
Reporting Bugs
Please take the time to read How to report bugs effectively (available in multiple languages)
For bugs in SME Server Release, SME Server Future & SME Server Translations the following format is suggested. It's not always appropriate so use some discretion, see eg. New Feature Requests
When reporting a new bug, and after choosing the platform, the next screen asks for more details on your problem. There are 4 main fields that need to be completed. These are Component, Found-In-Version, Summary, and Description. Fields such as Flags and Requestee should be left blank initially by most reporters. The CC field is for other people you would like notices on this issue to be sent to. YOU will always be automatically included by default, so you don't need to 'cc' yourself. The Attachment button at the bottom of the page is used to attach additional file information such as log files that may be helpful.
Component: As accurately as possible, you may choose which category your issue fits into. If Unknown then choose UNKNOWN.
Found-in-version: Usually self explanatory. If it applies to more than one version, choose the most recent stable release.
Summary: How would you describe the bug. BE BRIEF. A good summary should quickly and uniquely identify a bug report. It should explain the problem, not your suggested solution.
- Good: "Canceling a File Copy dialog crashes File Manager"
- Bad: "Software crashes"
- Bad: "Browser should work with my web site"
Description: Present the details of your problem in this section. You may use the following as a template:
ENVIRONMENT: Give a brief server history
eg. SME Server 7.1, Updated to 7.3 with yum
MODIFICATIONS: List any contribs or modifications you have on the server
eg. If the following produce any output, attach the results (don't post inline)
/sbin/e-smith/audittools/templates
/sbin/e-smith/audittools/newrpms
or say, no custom-templates or contribs.
ORIGINAL PROBLEM: State what you were trying to do.
STEPS TO REPRODUCE: As accurately and exactly as possible state what you did to get your problem.
ACTUAL RESULTS: List what you got as a result.
EXPECTED RESULTS: List what you expected to happen.
Attachment: This is used to attach (not post in line) any logs or results of command line queries such as the examples listed in "Modifications" above.
Finally, when satisfied with your entries, submit your bug report by pressing the COMMIT button.
Verifying Bugs
Bugs in the SME Server Release, SME Contribs & SME Server Translations products
need to be verified when fixed, see the Verification_Queue for the current list. See SME_Server:Documentation:QA:Verification for a detailed description of the verification process.
An example is bugzilla:3727 or bugzilla:7364.
Information from here is used in the Updates Announcements and if you use this format, it will save duplication of work
Other Products can be Closed after being fixed, as they are not part of the updates process.
The bug report will detail the exact package and version needed. These are normally in smeupdates-testing and can be installed by :
yum --enablerepo=smeupdates-testing update <package>
for example
yum --enablerepo=smeupdates-testing update e-smith-ldap
As it is a manual step to move packages into smeupdates-testing the most recent packages will be in smetest and can be installed from there if necessary. for example
yum --enablerepo=smetest update e-smith-ldap
or if you need several repositories :
yum --enablerepo=smetest,smeupdates-testing,smedev update e-smith-ldap
When performing verification, please use the following template:
VERIFICATION TEMPLATE
= ENVIRONMENT: = ORIGINAL PROBLEM: = RESOLUTION: = CURRENT VERSION INSTALLED: = TESTING: = UPDATED VERSION INSTALLED: = PROBLEM FIXED: = VERIFIED OR REOPEN: = DOCUMENTATION IMPACT: = SUGGESTED RELEASE NOTES:
AS AN EXAMPLE:
VERIFICATION = ENVIRONMENT: [description of test system (version, installation methods, upgrade history. etc). If you have some non-core package installed, run /sbin/e-smith/audittools/newrpms and provide output] = ORIGINAL PROBLEM: [Summarise bug] = RESOLUTION: [Insert changelog for new package] In example: Fixed in e-smith-base * Mon Apr 21 2013 chris burnat <devlist@burnat.com> 5.4.0-27.sme - Fix the way '.' works in bash [SME: 7532] = CURRENT VERSION INSTALLED: [rpm -qa <package name>] =TESTING: [Reproduce bug if you can, showing steps taken] = UPDATED VERSION INSTALLED: [Update to new package, show steps taken] and: [rpm -qa <package name>] = PROBLEM FIXED?: [Repeat steps carried out under TESTING above. = VERIFIED OR REOPEN: [Problem fixed, then VERIFIED - not fixed, then REOPEN] Note: if you may not have rights to toggle resolution, someone will do it for you = DOCUMENTATION IMPACT: [Does something need documentation, eg a db or new procedure? if affirmative, please provide details, someone will later punt to docteam for action] = SUGGESTED RELEASE NOTES: [Summary of what was fixed]
Testing
It is recommended that some basic tests be carried out before resolving VERIFIED. The scope of these tests very much depends on functionality associated with the package(s) under consideration. The following list whilst not fully inclusive can be used as a guide:
- Hosting http
- Hosting https
- Access external http
- Access external https
- SSH access to server
- Send Email
- Receive Email
- Use Webmail
- Use Samba
- FTP access to SME
- Check /var/log/messages or/and any other relevant log.
General
Bug Categories
Bugs are sorted by Product, they should be mostly self explanatory. Future/7.x/8.x/9.x may need clarifying.
With the release of SME 8.0, development is now focussed on reaching 9.0
- SME Server Future
- Bugs can be moved to future if they can be classified as NewFeatureRequest
- or as cleanup functionality that does not have a major (user-visible) impact.
- Bugs that should be fixed, but not sure which future release they will be fixed in
- SME Server Release 7.X
- The error is severe enough that we need to fix it, taking time away from 9.x development
- The bug is being worked on actively
- The bug has been cloned from 8.x for backporting
- SME Server Release 8.X
- The error is severe enough that we need to fix it, taking time away from 9.x development
- The bug is being worked on actively
- SME Server Release 9.X
- current development
Bug Triage
If you can help, the developers can concentrate on fixing the bugs.
The steps in this list take approx 90% of the time in the resolving of most bugs.
Reread How to report bugs effectively
Now look at the bug reports in Bugzilla with your developer hat on.
- Go into triage mode this page can help You : Bugzilla_Reports
- Ask the questions asked by the essay
- Get the information from the reporter
- Reproduce the issue
- Confirm the bug or mark it invalid if you can't reproduce it
- Summarise the issue if the report isn't clear
- Add a note about a workaround/fix if one is obvious to you
- That fix may not be the one we adopt, but it often helps if you tell us what you did to fix it
Bug Status
Resolution status:
- UNCONFIRMED
New bugs submitted by ordinary users are created in this state. Note. Project Developers tend not to look at unconfirmed bugs as they only have so much time, so it's important regular users check each others bugs.
- NEEDINFO
This is a new state that should be used when more information is needed by the bug reporter.
- CONFIRMED
Once bugs are confirmed to be valid and present in SME Server, by anyone see note above, change the status to CONFIRMED. Only change to CONFIRMED if the bug is well written and has enough information to proceed.
- IN_PROGRESS
This indicates a developer is actively working on the bug.
- FIXED
Set to fixed after the problem has been solved for others, eg a new rpm is available, documentation has been added, etc.
For SmeServer category bugs changing to FIXED is reserved for devs.
Occasionally the dev will forget to set the FIXED status and as part of the verification process will be pushed to FIXED on the way to VERIFIED by the person verifying the package. It should never be set to FIXED or VERIFIED without having a released package that is in CVS.
Bugs in Documentation, Translation, Contribs, or Websites categories need community involvement, anyone can resolve these FIXED.
Alternatives to Fixed are :
- NOTABUG
If you see that a problem is not a bug. eg User error Mark INVALID if the report is of documented (i.e. expected and correct) behaviour, or was encountered in some explicitly unsupported circumstance
- WONTFIX
This bug is:
- Too hard to fix
- Too rare
- Not something we can fix
- Something we chose not to expend resources on fixing
This is a hard call, usually made by the dev team. It may be a serious issue for the reporter but we may simply say "Sorry, that's just not worth the effort involved"
- LATER
Leave the bug for later, eg in case another bug will fix a related issue.
- REMIND
Needs follow-up from the reporter or someone else to get any further. Please REOPEN bugs when the follow-up comes in.
- WORKSFORME
You've been able to show that the problem does not happen for you. You should do this on a known good test box without any contribs installed. Many of our reports are breakage due to contribs.
- DUPLICATE
For when the reporter didn't find the related bug.
- VERIFIED
Check the fix provided solves the issue.
- CLOSED
Bug Tracker Administrators Close bugs in SME Current or SME Future,
but the reporter or maintainer can close in other sections, eg Documentation, Websites, Contribs
There is no need to verify a non-fix. Invalid/wontfix bugs can move straight to closed.
Bugzilla flags
These will be obvious to the people they are intended to serve. If you feel comfortable setting them based on information contained in the bug or needs you have for the bug then set them. If not then leave them be.
- need_help_to_verify [-+?]
"I don't understand how to verify this bug - please help" - a request to the dev team or other verifiers to have another look at the bug. Please say what you don't understand and what you've looked at.
- package_maintained_upstream
The package is maintained in an upstream repository. We won't be patching it locally. We've deemed it to be not important enough for us to fork the package, and so we'll wait for the fix to propogate from upstream. In these case we like to add a URL to the upstream bug report.
- patch_or_code_attached [-+?]
There is either a patch or some code attached to this bug. Attached may actually mean inline in the text, but the bug contains some code change which someone says works for them.
- Review [-+?]
This bug/change needs review by someone else. This is used by the dev team for "Do you think this change is a good one?"
- [-+?]
Set the flag to ? to say someone needs to look at it
