More than 3 years ago I’ve written an article about self-hosting email. Throughout the years that setup was adjusted, migrated and changed in various ways. I’ve used it on the FreeBSD, Debian, Ubuntu since the original article which was based on CentOS.

Now, I’m trying to set it up again on the CentOS, this time on the CentOS 8 and components to achieve the whole flow are changed a bit, and there are even some new ones.

Setup as a whole will be able to send emails, receive emails, scan incoming messages for spam, sign outgoing messages with DKIM and many more.

It is also worth mentioning that I’m hosting multiple domains on this server, so some settings will reflect that. Setup works just fine with one domain as it does with multiple domains, it is just that there are some additional configurations I’m doing that help achieve multi-tenancy.

Basic server setup

As with any new installation available on the internet I do some minor tweaks which help to protect the instance.

Disable password authentication

To achieve that following settings need to be adjusted in /etc/ssh/sshd_config:

PermitRootLogin without-password
PasswordAuthentication no

There are some other tweaks I do by using common Ansible role while deploying the server, but those steps aren’t really relevant for this article.

Package installation

There are few packages we need in order to have functioning setup. Some of them are available in the official repositories, while for some we need epel repository or custom repository in the case of rspamd.

EPEL repository

dnf install epel-release

Rspamd repository

curl https://rspamd.com/rpm-stable/centos-8/rspamd.repo > /etc/yum.repos.d/rspamd.repo
rpm --import https://rspamd.com/rpm-stable/gpg.key
dnf update
dnf install rspamd

Package installation

dnf install postfix dovecot redis

LetsEncrypt

On CentOS 8 there’s no RPM package (yet?), as there is for older releases and Debian/Ubuntu based systems. I miss that part as it provides both Systemd timer and service as well as the logrotate script for automatic log file rotation.

The only solution for now is to do all manually.

Installation

To install certbot executable manually use

wget -O /usr/local/bin/certbot https://dl.eff.org/certbot-auto

Also ensure proper permissions

chmod 0755 /usr/local/bin/certbot

Besides that I’ll also set up certbot.service to kick certificate renewal

[Unit]
Description=Certbot
[Service]
Type=oneshot
ExecStart=/usr/local/bin/certbot -q renew
PrivateTmp=true

And also the certbot.timer in order to start certbot renewal every so often

[Unit]
Description=Run certbot twice daily

[Timer]
OnCalendar=*-*-* 00,12:00:00
RandomizedDelaySec=43200
Persistent=true

[Install]
WantedBy=timers.target

This of course requires reload of the systemd to read new service and timer files which can be done with

systemctl daemon-reload

And to enable the timer to start automatically with the system

systemctl enable --now certbot.timer

When issuing and renewing certificates, certbot saves log files to the /var/log/letsencrypt folder, so we need cleanup that folder to prevent lint over time.

There are two options here. First one is to create /etc/letsencrypt/cli.ini file with

max-log-backups = 12

And if you prefer logrotate and have it installed, set that option to 0 and add following to your logrotate scripts

/var/log/letsencrypt/*.log {
    rotate 12
    weekly
    compress
    missingok

Issue certificates

Now with the whole thing set up it is time to issue certificates for the domain names used.

There are multiple combinations which can be used here. We can either have one certificate listing all of the domain names on the system or we can have separate certificate for each of the domains. I prefer the second option.

First option would resolve the SNI requirement as we would only serve one certificate which would match all domain names, but when adding new domain name this would require us to adjust current certificate and issue it again. It would also expose all of the configured domains for enumeration. With separate certificate for each domain those are not concerns, but in Dovecot we need to configure SNI options, and for Postfix we have to deal with a warning as SNI is not supported in currently installed version. It is discussed for future (3.7?) releases.

Anyhow, to issue a certificate we use standalone plugin as we don’t have web server to perform the challenge. With web server available I prefer webroot based verification

certbot certonly --standalone -d MYDOMAIN -m MYCONTACTEMAIL -n --agree-tos

Note: First certbot run will also bootstrap dependency installation via DNF and install required python packages.

This will give us certificates for configuration of other services.

Postfix

Once the Postfix is installed we need to configure it. Before doing any changes I like to backup the files I plan on changing so I have them for reference, but you do you.

cp /etc/postfix/main.cf{,.backup}
cp /etc/postfix/master.cf{,.backup}

main.cf

Changes I do in main.cf go into multiple sections and there are also some default configured for some of them, so I’ll just post the changes in the code blocks.

To expand a bit further on that. When you see lines starting with # that means that by default that option was enabled and I enabled it, while options without # are the settings I either added or changed existing.

Generic settings

myhostname = mail1.tomica.net
inet_interfaces = all
#inet_interfaces = localhost

Virtual domain configuration

# Virtual domain configuration
virtual_mailbox_domains = /etc/postfix/domains
virtual_mailbox_base = /var/mail
virtual_alias_maps = hash:/etc/postfix/virtual
virtual_transport = lmtp:unix:private/dovecot-lmtp

Settings basically mean that I’ll have the list of configured domains in /etc/postfix/domains, I’ll use /var/mail as home folder and alias maps should be in /etc/postfix/virtual hash database (postmap needs to be run on that file first). Incoming messages will then be transmitted to dovecot-lmtp socket. For system based aliases you can adjust /etc/aliases and add them to postalias

Format of the /etc/postfix/domains is basically one domain per-line

DOMAIN1
DOMAIN2
DOMAIN3

Format of the /etc/postfix/virtual is as follows

# Alias DOMAIN3 to DOMAIN1
@DOMAIN3 @DOMAIN1

# Alias ADDRESS to ADDRESS 1 and 2 at DOMAIN1
ADDRESS@DOMAIN1 ADDRESS1@DOMAIN1, ADDRESS2@DOMAIN1

This of course requires hash table generation which can be achieved with

postmap /etc/postfix/virtual

And to hash system aliases file you can

postalias /etc/aliases

TLS settings

# TLS hardening
tls_random_source = dev:/dev/urandom
smtpd_tls_ciphers = high
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
smtpd_tls_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
lmtp_tls_mandatory_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
lmtp_tls_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
smtp_tls_mandatory_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
smtp_tls_protocols = TLSv1.2,TLSv1.1,TLSv1,!SSLv3,!SSLv2
smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CBC3-SHA, KRB5-DES, CBC3-SHA
smtpd_tls_dh1024_param_file = /etc/postfix/dhparams.pem
smtp_tls_note_starttls_offer = yes

These settings basically adjust default SMTP and SMTPD protocol settings (incoming and outgoing connections) as well as the LMTP transport (although not so important as we use socket in this particular case).

And to have valid SSL certificate for incoming connections (when clients connect to the Postfix) we need to configure TLS certificate

smtpd_tls_cert_file = /blah
smtpd_tls_key_file = /blah.key

We’re also generating custom dhparams.pem file for Postfix in order to avoid Weak DH and Logjam. To do that I use:

openssl dhparam -out /etc/postfix/dhparams.pem 2048

Simple authentication and security layer

Or for short: SASL.

# SASL
smtpd_sasl_type = dovecot
broken_sasl_auth_clients = yes
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination
smtpd_relay_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination

Message size limit

Default message size limit in Postfix is 10MB, which is in this day and age a bit low. I prefer to keep it at 50MB

# Message size limit
message_size_limit = 51200000

Rspamd postfix configuration

And finally, to wrap up Postfix’s main configuration we add the rspamd milter. It will be configured on localhost:11332 as we’ll use proxy worker, but more on that later.

In scenario where milter is down (rspamd is down) we accept the email. This seems like a sane action to me as I don’t want to lose the incoming email if scanner fails for whatever reason.

# Rspamd
smtpd_milters = inet:localhost:11332
milter_default_action = accept # Accept if check fails

master.cf

With main configuration out of the way, we also configure listeners in the master.cf. For that it is sufficient to enable following blocks:

submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING
smtps     inet  n       -       n       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_client_restrictions=$mua_client_restrictions
  -o smtpd_helo_restrictions=$mua_helo_restrictions
  -o smtpd_sender_restrictions=$mua_sender_restrictions
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

This will enable listeners on ports 465 and 587, the ones that listen on the secure ports, because the default is to listen just on port 25.

Dovecot

With Postfix configured we can continue to Dovecot configuration. Package is structure in a way that configuration is spitted by subsystem, each having it’s own separate file. So as with Postfix I first backup all of the files I intend to change and then perform the changes:

cd /etc/dovecot/conf.d/
for FILE in 10-auth.conf 10-mail.conf 10-master.conf 10-ssl.conf ; do cp -a $FILE $FILE.backup ; done

With that out of the way, let’s now change those files.

10-auth.conf

Authentication system by default uses system username/password combinations for login. I don’t want that, instead, I hash the passwords and keep them in flat file, so for that I need passwd file auth plugin

#!include auth-system.conf.ext
!include auth-passwdfile.conf.ext

10-mail.conf

This file describes where to store the email, what user/group combination should access/own those files and which privileged group to use during that process.

Mail is being kept in /var/mail/DOMAIN/USER folder on-disk.

I’ve opted to use system user and group “mail” which is present by default, and since Dovecot tries to prevent shooting yourself in the foot by specifying root or some daemon user for storing emails it has limitation by default that mail uid and gid have to be above 500, so we also need to override that part.

mail_home = /var/mail/%d/%n
mail_location = maildir:~
mail_uid = 8
mail_gid = 12
mail_privileged_group = mail
first_valid_uid = 8

10-master.conf

To use same credentials for Postfix (SMTP) as we use for Dovecot (POP/IMAP) we expose authentication listener to the postfix and allow r/w permissions to it. Postfix is already configured as you can see above (smtpd_sasl_path = private/auth) and now we just need to place socket there. We do the same for transporting messages to dovecot via LMTP (virtual_transport = lmtp:unix:private/dovecot-lmtp)

service lmtp {
# ...
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0660
    user = postfix
    group = mail
  }
# ...
}
service auth {
# ...
  unix_listener auth-userdb {
    mode = 0660
    user = mail
    group = mail
  }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = mail
  }
# ...
}

10-ssl.conf

Finally, for the SSL/TLS settings we specify default certificate being served, but we also configure SNI support, so each domain is getting its own SSL certificate.

Unfortunately SNI option is not available in current Postfix version, so security warning will pop up if you configure email client to use something else than server’s domain name for sending email.

ssl_cipher_list = ALL:!LOW:!MEDIUM:!SSLv2:!SSLv3:!EXP:!aNULL:+HIGH
ssl_prefer_server_ciphers = yes

local_name YOURDOMAIN {
        ssl_cert = </etc/letsencrypt/live/YOURDOMAIN/fullchain.pem
        ssl_key = </etc/letsencrypt/live/YOURDOMAIN/privkey.pem
}

Rspamd

In the past I’ve used combination of Amavis-new, SpamAssassin, OpenDKIM to achieve functionality that this wonderful piece of software has all built in. Looking at the official project site as well as the documentation, it is visible that this is intended to be quite polished and enterprise-ready utility. It offers many advanced features and options which include high-availability, various caching mechanisms, Redis in-memory database support for storing some settings and so on. This software could easily have it’s own book, but documentation is so simple and to-the-point that it doesn’t make sens to dwell on it much here.

There’s also the fact that setup I’m using is so simple that you might just wonder if it is really needed.

Redis

In Rspamd Redis is used for storing both volatile and non-volatile data such as:

  • tokens storage
  • cache of learned messages by statistical module (bayes)
  • fuzzy storage backend
  • K/V cache storage for many rspamd modules
  • greylisting (delaying suspicious emails)
  • rate-limiting
  • whitelisting reply messages (stores reply message ID to avoid certain checks for replies to our own sent messages)

Official documentation recommends separate redis instance for each module that stores non-volatile data + maxmemory limit and policy which purges oldest keys under memory pressure.

Again, this setup is simple, so I just configure single Redis instance. This is being handled for me via Ansible but here are the relevant settings

Systemd unit file:

[Unit]
Description=Advanced key-value store
After=network.target
Documentation=http://redis.io/documentation, man:redis-server(1)

[Service]
Type=forking
ExecStart=/usr/bin/redis-server /etc/redis/redis-rspamd-main.conf
PIDFile=/var/run/redis/redis-rspamd-main.pid
TimeoutStopSec=0
Restart=always
User=redis
Group=_rspamd
ExecStop=/bin/kill -s TERM $MAINPID
UMask=007
PrivateTmp=yes
LimitNOFILE=65535
PrivateDevices=yes
ProtectHome=yes
ReadOnlyDirectories=/
ReadWriteDirectories=-/var/lib/redis-rspamd-main
ReadWriteDirectories=-/var/log/redis
ReadWriteDirectories=-/var/run/redis
CapabilityBoundingSet=~CAP_SYS_PTRACE

# redis-server writes its own config file when in cluster mode so we allow
# writing there (NB. ProtectSystem=true over ProtectSystem=full)
ProtectSystem=true
ReadWriteDirectories=-/etc/redis

[Install]
WantedBy=multi-user.target
Alias=redis-rspamd-main.service

Note the usage of Group _rspamd, this sets up socket ownership in a way that rspamd is able to write to it and read from it.

Main configuration, relevant parts:

port 0
bind 127.0.0.1
unixsocket /var/run/redis/redis-rspamd-main.sock
unixsocketperm 770
logfile /var/log/redis/redis-rspamd-main.log
dir /var/lib/redis-rspamd-main

Be sure all folders (run, log, lib) are created and have proper ownership (redis)

Configuration

To perform basic configuration I use configuration wizard provided by rspamadm command:

rspamadm configwizard

It will ask you multiple Y/n questions where capital one is the default. I’ll list my answers below:

Setup WebUI and controller worker:
Controller password is not set, do you want to set one?[Y/n]: Y
Enter passphrase: BLAHBLAHBLAHmyPassword
The following modules will be enabled if you add Redis servers:
	* greylist
	* url_redirector
	* replies
	* neural
	* ratelimit
	* history_redis
Do you wish to set Redis servers?[Y/n]: Y
Input read only servers separated by `,` [default: localhost]: /var/run/redis/redis-rspamd-main.sock
Input write only servers separated by `,` [default: localhost]: /var/run/redis/redis-rspamd-main.sock
Do you have any password set for your Redis?[y/N]: n
Do you have any specific database for your Redis?[y/N]: n
Do you want to setup dkim signing feature?[y/N]: Y
How would you like to set up DKIM signing?
1. Use domain from mime from header for sign
2. Use domain from SMTP envelope from for sign
3. Use domain from authenticated user for sign
4. Sign all mail from specific networks

Enter your choice (1, 2, 3, 4) [default: 1]: 3
Allow data mismatch, e.g. if mime from domain is not equal to authenticated user domain? [Y/n]:
Do you want to use effective domain (e.g. example.com instead of foo.example.com)? [Y/n]: n
Enter output directory for the keys [default: /var/lib/rspamd/dkim/]:
Enter domain to sign: MYDOMAIN
Enter selector [default: dkim]:
Do you want to create privkey /var/lib/rspamd/dkim/mydomain.example.dkim.key[Y/n]: Y
To make dkim signing working, you need to place the following record in your DNS zone:
dkim._domainkey IN TXT ( "v=DKIM1; k=rsa; "
	"p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDRHkuW5hfF7BkwbWHXWSQHCeEhly1oFCBoiEEH3n2db+x9ljpeWhkISmJiGQRAz6h4nlCYkY37FzkQaXXB1eqze7apkV6tQBJ7WVCRo43QMPQvK3wT8vH7j0X2D8da6xgleHr4I1K+sZQ/fgBdMacqGRxD0eNLiAJsabP0G5cdwQIDAQAB" ) ;
Do you wish to add another DKIM domain?[y/N]:
...

At the end it will print out all configured options and and ask you to apply them:

Apply changes?[Y/n]: Y

To have settings in-place you need to restart/reload rspamd service. But I’ll leave that for the end, after few minor things are adjusted.

It also prints the DKIM key you should place in your DNS zone during the DKIM key generation. This key can later be found in the pub file in /var/lib/rspamd/dkim/

Disable normal worker

As it is not used in self-scan mode place the following in /etc/rspamd/local.d/worker-normal.inc

enabled = false; # not needed in self-scan mode

Configure proxy worker

In our scanning we’re configuring proxy worker which will do the self-scan, will enable milter mode and will automatically place spam header in message headers. I have the following in /etc/rspamd/local.d/worker-proxy.inc

milter = yes; # Enable milter mode
timeout = 120s;

upstream "local" {
        default = yes; # Self-scan upstreams are default
        self_scan = yes;
}

count = 4; # no. of processes in self-scan mode
max_retries = 5;
discard_on_reject = false;
quarantine_on_reject = false;
spam_header = "X-Spam";
reject_message = "Spam message rejected"

Setup DKIM for additional domains

In order to setup new domain name for DKIM signing we need to first generate private/public key combination which can be achieved by the following command with appropriate domain (-d) and selector (-s)

rspamadm dkim_keygen -s 'dkim' -d MYOTHERDOMAIN -k /var/lib/rspamd/dkim/MYOTHERDOMAIN.dkim.key | tee /var/lib/rspamd/dkim/MYOTHERDOMAIN.dkim.key.pub

We also have to enable domain by adding it to the domain list in /etc/rspamd/local.d/dkim_signing.conf

domain {
    MYDOMAIN {
        selector = "dkim";
        path = "/var/lib/rspamd/dkim/MYDOMAIN.dkim.key";
    }
}

Alternative, and most likely better way, is to set up DKIM maps as explained in official documentation.

My settings for multiple domains have following appended to the /etc/rspamd/local.d/dkim_signing.conf

selector_map = "/etc/rspamd/dkim_selectors.map";
path_map = "/etc/rspamd/dkim_paths.map";

Selector map basically determines which selector is used for domain signing

DOMAIN dkim

While the path map determines which key is used for signing particular domain

DOMAIN /var/lib/rspamd/dkim/DOMAIN.key