HPKP Public Key Pinning

A good way to protect your site from fraudulent certificates is to implement HTTP Public Key Pinning a.k.a HPKP. This lets your visitor’s browsers know which certificate they should be expecting from your site. If it receives a certificate it doesn’t expect, the connection will fail.

Implementing this is quite easy as we’re just adding a header to the configuration. To get started we’ll need a few things in place.

We’ll need to decide which certificates we are going to pin. You need to include at least one certificate you ARE using and at least one certificate you are NOT using. The certificate you are not using is your backup certificate. This way, if the certificate you are using and pinning gets revoked or otherwise becomes unusable (you loose the private key, for example), you have a get out of jail free card.

As mentioned, you need to pin at least one certificate you are using. You don’t have to pin your server’s certificate, you are able to pin your certificate authority’s intermediate certificate. This has the advantage of letting you revoke or regenerate your server’s certificate without having to update your HPKP header. This is the method I initially employed after switching to Let’s Encrypt. A good explanation of various pinning methods can be found over on Scott Helme’s blog. [1]

Let’s get started. First we need to make sure the headers module is enable in Apache:

# a2enmod headers

There are a couple of ways to get your certificate’s base64 encoded public key. The easiest would be to head over to SSL Labs and test your site, on that page you will find the “Pin SHA256” for each certificate your server is sending out.

You can also extract this string from a certificate, private key or certificate signing request: [2]

From a certificate:

openssl x509 -in my-certificate.crt -pubkey -noout | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64

From your private key:

openssl rsa -in my-key-file.key -outform der -pubout | openssl dgst -sha256 -binary | openssl enc -base64

From your certificate signing request:

openssl x509 -in my-certificate.crt -pubkey -noout | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64

If we wanted to get the public key from Let’s Encrypt’s X3 Intermediate Authority, we can do the following:

# wget -q https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem -O - | openssl x509 -pubkey -noout | openssl rsa -pubin -outform der | openssl dgst -sha256 -binary | openssl enc -base64
writing RSA key

We now have the first of our two pins. To get the second, we can generate our own backup certificate. You can of course use any certificate you are not actively using:

# openssl req -nodes -newkey rsa:2048 -keyout example.key -out example.csr
Generating a 2048 bit RSA private key
writing new private key to 'example.key'
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:CA
State or Province Name (full name) [Some-State]:Ontario
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:www.example.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

We now have our backup certificate generated, we can get it’s pin from the private key:

# openssl rsa -in ./example.key -outform der -pubout | openssl dgst -sha256 -binary | openssl enc -base64
writing RSA key

With that we have everything we need to create our header. Edit the config file for your site, e.g. /etc/apache2/sites-enabled/default-ssl.conf and add the following anywhere within virtualhost (substituting your own pins of course). I put mine right next to my Strict-Transport-Security header:

Header always set Public-Key-Pins 'pin-sha256="YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg="; pin-sha256="zjL4m+14ZUqqAj3No4frEbwv3oMsjQabh1LSDXF0efI="; max-age=5184000; includeSubdomains;'

This will set your header to pin Let’s Encrypt’s intermediate certificate and your backup certificate in a browser for two months which is “considered a balance between the two competing security concerns.” [3]

Be sure to reload apache’s configuration:

# service apache2 reload

And then head over to SSL Labs to make sure you have everything setup correctly.


[1] https://scotthelme.co.uk/guidance-on-setting-up-hpkp/
[2] https://developer.mozilla.org/en/docs/Web/Security/Public_Key_Pinning
[3] https://tools.ietf.org/html/rfc7469#section-4.1

Enabling HTTP2 on Apache under Ubuntu 16.04

I reciently upgraded to Ubuntu Server 16.04 and was bummed to find out that HTTP2 support had been removed from Apache. Luckily we can easily add the plug-in and enable it. [1]

To do this, we need to build Apache ourselves and then copy out the one file we need. Let’s make sure we have the prerequisites:

apt-get install devscripts build-essential fakeroot

Now install libnghttp2 and Apache:

apt-get install libnghttp2-dev apache2

Now we build Apache from source. Check to make sure these three lines are added and not commented from /etc/apt/sources.list:

deb-src http://archive.ubuntu.com/ubuntu xenial main restricted universe
deb-src http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe
deb-src http://security.ubuntu.com/ubuntu xenial-security main restricted universe multiverse

And update for good measure:

apt-get update

Now we’re ready to begin:

cd /tmp
mkdir apache2
cd apache2
apt-get source apache2
apt-get build-dep apache2
cd apache-2.4.18
fakeroot debian/rules binary

Now we can move mod_http2.so to Ubuntu’s Apache installation:

cp /tmp/apache2/apache-2.4.18/debian/apache2-bin/usr/lib/apache2/modules/mod_http2.so /usr/lib/apache2/modules/

Next we create the load file:

nano /etc/apache2/mods-available/http2.load

And paste this line:

LoadModule http2_module /usr/lib/apache2/modules/mod_http2.so

Next edit your site’s .conf file to include:

Protocols h2 http/1.1

Now we can enable http2 and restart apache:

a2enmod http2
service apache2 restart

That’s it! For cleanup, you can remove the /tmp/apache2 folder and remove the prerequisites if you no longer need them:

rm -Rf /tmp/apache2
apt-get remove devscripts build-essential fakeroot
apt-get autoremove

1. https://zitseng.com/archives/10470

Updated SSLCipherSuite String

Chrome has been complaining “Your connection to website is encrypted with obsolete cryptography”.  I made a change to my SSLCipherSuite string located at /etc/apache2/mods-available/ssl.conf to fix this.

Chrome would like you to be using anything with a higher hash than SHA1 and using GCM instead of CBC suites.[1]  For a simple fix, we can move the one Chrome currently prefers to the top of the list:


This will be fine until Chrome (and other browsers) support AES256-GCM-SHA384. If you don’t mind a longer string and would like to future-proof now, you can change your string to:


Make sure you have SSLHonorCipherOrder set to on:

SSLHonorCipherOrder on

[1] http://security.stackexchange.com/a/83891

Adding DKIM Signing Into the Mix

It was pointed out that I neglected to include DKIM signing in this tutorial as I promised in a previous post. Let’s fix this oversight now.

Since we already have a mail server up and running, we can easily sign our outgoing messages with DKIM to help prevent forged messages and authenticate legitimate ones.  For more information on DKIM, look here.

Before we get started, make sure you can edit the DNS records for your domain. We will need to add a TXT record a little later to make DKIM work.

First let’s install OpenDKIM: [1]

apt-get install opendkim opendkim-tools

To configure OpenDKIM, first we’ll edit /etc/opendkim.conf:

Syslog                  yes

Domain                  example.com
KeyFile                 /etc/opendkim/110414.private
Selector                110414

AutoRestart             yes
Background              yes
Canonicalization        relaxed/relaxed
DNSTimeout              5
Mode                    sv
SignatureAlgorithm      rsa-sha256
SubDomains              no
X-Header                no

You can make the Selector line anything you like. I prefer to make it the date I generated the DKIM key, that way it it’s easier to change later. Some people make it “mail” or the name of their company or organization.

Edit /etc/default/opendkim and add/edit/uncomment the following line (making sure it is the only uncommented line):

SOCKET="inet:8891@localhost" # Ubuntu default - listen on loopback on port 8891

Next we tell Postfix to use OpenDKIM. Edit /etc/postfix/main.cf and add the following lines:

milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Now we can create our DKIM keys. Make note of what you made your Selector and run the following using your own domain name:

cd /etc/opendkim
opendkim-genkey -b 2048 -s <your Selector, e.g. 110414> -d <your domain name, e.g. example.com>

You should now have two files, using this example you would have 110414.txt and 110414.private.

110414.txt contains the information we need to add to DNS. If we take a look at this file:

cat 110414.txt

It will look something like this:

110414._domainkey       IN      TXT     ( "v=DKIM1; k=rsa; "
          "zf4ZHf/BSih0ZZaFbo9sBei96JIIzGTqQZEWCUTSMkzZsKcHOQLs8L+r5eYwDwpxdVtFByzgrN56WVB7IYMhDByVOGntJLQ1vRMbfg6RcA9Ezv7dsCndkXGWWcEb9KISOO1ozTwwIDAQAB" )  ; ----- DKIM key 110414 for example.com

You will need to add this as a TXT record for your domain. The steps for doing this will differ depending on who is hosting your DNS. The basics are, you need to add a TXT record for <your Selector>._domainkey.example.com containing everything between the two brackets.

Head over to DKIMCore to test your DNS record and make sure it is configured properly.

Once your public key is installed in DNS and verified with DKIMCore, we can restart OpenDKIM and Postfix:

service opendkim restart
service postfix restart

Now send yourself a test email. If you take a look at the headers of your message, you should see something similar to:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=example.com; s=110414;
	t=1415142449; bh=t2OTK1f3BCnxdQW7LVf81uHqulsjZAP9q/Ux4XDzhVw=;

If you see more than one signature on your message, you can fix this by editing /etc/postfix/master.cf. Find the line:

smtp      inet  n       -       -       -       -       smtpd

and add or append:

 -o receive_override_options=no_milters

If you followed my previous tutorial installing SpamAssassian, these lines would now look like this:

smtp      inet  n       -       -       -       -       smtpd
 -o content_filter=spamassassin
 -o receive_override_options=no_milters

Remember to leave a space before -o!

Now we can test to make sure DKIM signing is working for the outside world. The folks over at Port25 have a great tool for testing DKIM signatures. Simply send an email to check-auth@verifier.port25.com and wait for their response.

You should see this in their reply:

Summary of Results
SPF check:          pass
DomainKeys check:   neutral
DKIM check:         pass
Sender-ID check:    pass
SpamAssassin check: ham

1. https://help.ubuntu.com/community/Postfix/DKIM

Adding SSL/TLS Encryption to Postfix, Apache & Zarafa

SSL/TLS encryption can seem overly complicated. While this is true of the actual cryptography, it’s quite easy to implement.

We’ll be working with free StartSSL certificates. StartSSL certificates are recognized by all browsers, operating systems and mobile devices that I have tried. This solves any of the “not trusted” error messages that can appear when using self signed certificates.

Before we begin we need to make sure that your server has a sufficient source of randomness to accommodate encrypted connections.  Since we’re using a VPS and don’t have physical access to the machine, we can use haveged:

apt-get install haveged

You will also need to make sure that you have access to either the postmaster@, webmaster@ or hostmaster@ email addresses for the domain name you will be using.  StartSSL uses these accounts for verification when obtaining your certificate. If mail for your domain is already being delivered to your VPS you can setup these addresses to go to your account using an alias:

nano /etc/aliases

In this file, add the aliases you need, followed by a colon and your username:

postmaster: <username>
hostmaster: <username>
webmaster: <username>

Each time you modify the aliases file, you need to run:



Getting your certificate

Head over to StartSSL and sign up for an account.  Once you are signed up and logged in, select the Validations Wizard and then select Domain Name Validation.  Here you will enter your domain name.  It will then send an email with a validation code to either postmaster@, hostmaster@ or webmaster@ your domain name.  Once you receive this email, enter the code to complete the validation. [1]

Now we can generate a Certificate Signing Request to submit to StartSSL:

mkdir /etc/apache2/ssl
cd /etc/apache2/ssl
openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr

Select Certificates Wizard and then select Web Server SSL/TLS Certificate. The next step will ask if you want to generate a private key. Since we have already done that by generating our CSR, select Skip. Paste your CSR in the box and select Continue. Choose the domain the certificate is for & select Continue.

The next step will ask you to add a subdomain.  You most likely want to add www here, this way your visitors can visit https://www.yourdomain.tld.  Caution: Because we are also running a mail server, we need to make sure the name on our certificate matches the hostname of our droplet.  As long as your droplet has a hostname of www.yourdomain.tld or just yourdomain.tld you won’t run into any issues with a mismatched hostname.

Press Continue to complete the process.

Your certificate will appear in a text field. Lets copy our new certificate and paste it into a new file:

nano /etc/apache2/ssl/server.pem

Paste your certificate into this file and exit saving your changes.


Installing your certificate

Next, we’re going to create the certificate chain file that apache will use. It contains your certificate, and StartSSL’s intermediate certificate. (Including StartSSL’s root cert is not necessary because browsers already include it.)  Download the intermediate certificate from StartSSL:

wget https://www.startssl.com/certs/sub.class1.server.ca.pem

Now combine our new certificate with StartSSL’s intermediate certificate:

cat server.pem sub.class1.server.ca.pem > server-bundle.pem

You should now have five files:

ls -1 /etc/apache2/ssl

Let’s protect these files:

chmod -R 400 /etc/apache2/ssl

Before we get too far, now is a good time to back these files up. I would recommend archiving them and storing them somewhere other than your VPS:

tar -czf ~/sslcertbackup.tar.gz /etc/apache2/ssl/*

Log into your server using WinSCP (or equivalent) and download this file to store somewhere safe.

Now the fun part! Let’s use our new certificate to enable encryption in apache:

Enable some plugins in apache:

a2enmod ssl
a2enmod headers
a2enmod rewrite

And enable the apache’s default SSL site:

a2ensite default-ssl

Now we edit the default-ssl file to add our certificate:

nano /etc/apache2/sites-enabled/default-ssl.conf

Find these two lines and point them to your certificate:

SSLCertificateFile    /etc/apache2/ssl/server-bundle.pem
SSLCertificateKeyFile /etc/apache2/ssl/server.key

To add Z-Push to the SSL site, add these lines to the end of the file, just before <VirtualHost>:

Alias /Microsoft-Server-ActiveSync /usr/share/z-push/index.php
<Directory /usr/share/z-push>
php_flag magic_quotes_gpc off
php_flag register_globals off
php_flag magic_quotes_runtime off
php_flag short_open_tag on

Restart apache to load the plugins and apply our changes:

service apache2 restart

Now to enable encryption in Zarafa:

nano /etc/zarafa/gateway.cfg

To enable IMAPS and/or POP3S change these lines:

pop3_enable = no
pop3s_enable = yes
imap_enable = yes
imaps_enable = yes

Remember: We need to leave unencrypted imap enabled as this is what our SASL daemon uses.

And point these lines to our certificate:

ssl_private_key_file = /etc/apache2/ssl/server-bundle.pem
ssl_certificate_file = /etc/apache2/ssl/server.key

To add our certificate to the Zarafa iCal gateway:

nano /etc/zarafa/ical.cfg

Disable unencrypted iCal and enable encrypted iCal:

ical_enable = no
icals_enable = yes

And point these lines to our certificate:

ssl_private_key_file = /etc/apache2/ssl/server-bundle.pem
ssl_certificate_file = /etc/apache2/ssl/server.key

Restart these two services to apply the changes:

service zarafa-gateway restart
service zarafa-ical restart

Now to configure Postfix to use our certificate: [2]

postconf -e 'smtp_tls_security_level = may'
postconf -e 'smtp_tls_ciphers = medium'
postconf -e 'smtpd_tls_security_level = may'
postconf -e 'smtpd_tls_ciphers = medium'
postconf -e 'smtpd_tls_auth_only = no'
postconf -e 'smtp_tls_note_starttls_offer = yes'
postconf -e 'smtpd_tls_cert_file = /etc/apache2/ssl/server-bundle.pem'
postconf -e 'smtpd_tls_key_file = /etc/apache2/ssl/server.key'
postconf -e 'smtpd_tls_loglevel = 1'
postconf -e 'smtpd_tls_received_header = yes'
postconf -e 'smtpd_tls_session_cache_timeout = 3600s'
postconf -e 'tls_random_source = dev:/dev/urandom'
postconf -e 'myhostname = server.yourdomain.tld' # set this to match your Digital Ocean hostname

Now that we have encryption enabled, we can enable port 587 for message submission. For this we need to edit master.cf:

nano /etc/postfix/master.cf

Now uncomment the line:

submission inet n      -       n       -       -       smtpd

As always, reload Postfix to apply our changes:

service postfix reload


[Optional] Redirect HTTP to HTTPS

To disable unencrypted access to webmail and Z-Push:

a2dissite 000-default

To redirect unencrypted connections to HTTPS, we’ll create a redirect site:

nano /etc/apache2/sites-available/redirect.conf

Paste these lines into the file and update it to match your domain name:

    ServerName www.yourdomain.tld
    ServerAlias yourdomain.tld
    Redirect permanent / https://www.yourdomain.tld/

Now enable the site and reload apache:

a2ensite redirect
service apache2 reload


[Bonus] Setting up Forward Secrecy, OCSP Stapling & Strict Transport Security:

Forward secrecy has become very important as of late.  We can enable it with just a few simple modifications.

First, we’ll configure apache:

nano /etc/apache2/mods-enabled/ssl.conf

Find the line SSLCipherSuite and change it to the following. Add SSLHonorCipherOrder on just above:

SSLHonorCipherOrder on

These lines set Elliptic Curve ciphers first, followed by the slower Diffie Hellman ciphers. Both of these cipher sets offer forward secrecy. Finally we add a legacy 3DES cipher at the end for compatibility with older clients. 3DES is preferable over RC4 for security. [3]

Enable a stronger elliptic curve:

openssl ecparam -name secp384r1 >> /etc/apache2/ssl/server-bundle.pem

Caution: Note the double >>.  Using only one will overwrite your certificate file.

Next we enable forward secrecy in Postfix. First we generate our own DH parameters: [4]

openssl dhparam -out /etc/postfix/dh512.pem 512
openssl dhparam -out /etc/postfix/dh1024.pem 1024

Next we add them to Postfix and ensure EECDH is set to strong:

postconf -e 'smtpd_tls_dh512_param_file = /etc/postfix/dh512.pem'
postconf -e 'smtpd_tls_dh1024_param_file = /etc/postfix/dh1024.pem'
postconf -e 'smtpd_tls_eecdh_grade = strong'

And reload Postfix:

service postfix reload

Now that we have forward secrecy enabled. Lets move on to OCSP Stapling and Strict Transport Security: [5][6]

nano /etc/apache2/sites-enabled/default-ssl.conf

Add this line near the top of your file around DocumentRoot:

Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"

Add these lines around SSLCertificateFile:

SSLUseStapling on
SSLStaplingResponseMaxAge 43200
SSLStaplingCache "shmcb:logs/stapling-cache(150000)"

And restart apache:

service apache2 restart

Now head over to Qualys’s SSL Labs to test your server.  With all of these settings in place, you should receive an A+ rating.

You can also check to make sure encryption is working on your mail server by going over to CheckTLS.

As always, if you notice any errors, omissions or have anything to add that would make this tutorial better, please let me know in the comments below.

1. https://konklone.com/post/switch-to-https-now-for-free#register-with-startssl
2. https://help.ubuntu.com/community/Postfix
3. https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-broken-now-what
4. http://www.postfix.org/FORWARD_SECRECY_README.html#quick-start
5. https://www.grc.com/securitynow.htm#453
6. https://www.grc.com/securitynow.htm#412

Adding Spam Protection to Postfix

If you’ve followed my first tutorial, you already have a fully functional email server.  Unfortunately, where there is email, there is also spam.  We can easily add some protection against unwanted messages using some tools in Postfix as well as adding SpamAssassin, SPF checking and optionally, greylisting.

First lets install SpamAssassin: [1]

apt-get install spamassassin spamc libmail-dkim-perl

Now lets add a new user for SpamAssassin, and create a few directories for it to use:

groupadd spamd
useradd -g spamd -s /bin/false -d /var/log/spamassassin spamd
mkdir /var/log/spamassassin
chown spamd:spamd /var/log/spamassassin

Now we can configure SpamAssassin:

nano /etc/default/spamassassin

Replace ENABLED=0 with ENABLED=1 and replace CRON=0 with CRON=1.

Next, create a new line:


Now, find the OPTIONS= line and replace it with:

OPTIONS="--create-prefs --max-children 2 --username spamd -H ${SAHOME} -s ${SAHOME}spamd.log"

Exit nano by pressing CTRL-X and press Y to save your changes.

Now lets configure SpamAssassin’s rules:

nano /etc/spamassassin/local.cf

Edit and uncomment the following lines to look like: [2]

report_safe 0
required_score 3.0
use_bayes 1
bayes_auto_learn 1

Now add the line:

skip_rbl_checks 1

This will stop SpamAssassin from checking blacklist servers, as we will be adding this to Postfix a little later.  There is no point in checking twice as it adds unnecessary extra processing on your server and wastes bandwidth for the blacklist providers.

Exit nano saving your changes.  Now we can (re)start SpamAssissin:

service spamassassin restart

Next we’ll be configuring Postfix to use SpamAssissin:

nano /etc/postfix/master.cf

The first uncommented line should look like:

smtp     inet     n     -     -     -     -     smtpd

Add -o content_filter=spamassassin just below that line, so it looks like:

smtp     inet     n     -     -     -     -     smtpd
  -o content_filter=spamassassin

Notice the space before -o? You need to make sure there is a space between smtpd and the lines you have just added, otherwise Postfix will report an error.

Page down to the bottom of the file and add these lines:

spamassassin     unix     -     n     n     -     -     pipe
  user=spamd argv=/usr/bin/spamc -f -e
  /usr/sbin/sendmail -oi -f ${sender} ${recipient}

Exit nano and save your changes.

Now restart Postfix to apply your changes:

service postfix restart

Now is a good time to stop and test your changes.  Send an outbound email and send an incoming email to your server.  You can check the headers of the messages you send a receive to make sure SpamAssassin is adding headers to you messages.  You can also check the SpamAssassin log file:

nano /var/log/spamassassin/spamd.log


Adding DNSBL (Blacklists), Rate Limiting & Other Protection

Next we’re going configure Postfix to check two blacklists before receiving messages, as well as adding some other protection.  We go back to configuring Postfix:

nano /etc/postfix/main.cf

Find the line smtpd_recipient_restrictions and edit it to look like: [3]

smtpd_recipient_restrictions =
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client bl.spamcop.net,

And add the following lines: [4] [5]

disable_vrfy_command = yes
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname
strict_rfc821_envelopes = yes
smtpd_error_sleep_time = 1s
smtpd_soft_error_limit = 10
smtpd_hard_error_limit = 20
smtpd_client_connection_count_limit = 10
smtpd_client_connection_rate_limit = 60

These lines add checks to Postfix, making sure that the hostnames connecting to your mail server actually exist and are valid.  As mentioned before, we have also added in two blacklist checks here.  This prevents sites that have been blacklisted from even sending their messages to your server, saving processing power and bandwidth.

The two blacklists I have selected are the Spamhaus Zen list and Spamcop‘s list.  Spamhaus seems to block the majority of spam and SpamCop blocks most of the rest. If something does make it through, you are able to report these messages to SpamCop and they will be blocked in the future.  Zarafa is configured by default to put any message that is marked as spam in the Junk E-Mail folder.

We’ve also added rate limiting here, so that spamming mail servers don’t flood your server with connections.

Blacklists are a great and effective way of blocking a large amount of spam.  Occasionally a legitimate email server can find itself added to blacklist, this can happen for a variety of reasons.  You can request to be added to a whitelist which can help this problem.  You can also create your own whitelist with servers you know you can trust.

To create our own whitelist: [6]

nano /etc/postfix/rbl_override

Add the IP addresses or hostnames of mail servers you trust: OK OK
mail.importantclient.com OK

Exit and save your changes. Now run: (You will also need to run this command each time you modify this file)

postmap /etc/postfix/rbl_override

Now we tell Postfix to use this whitelist:

nano /etc/postfix/main.cf

We will need to add the line check_client_access hash:/etc/postfix/rbl_override to smtpd_recipient_restrictions after reject_unauth_destination and just before the blacklist checks:

smtpd_recipient_restrictions =
  check_client_access hash:/etc/postfix/rbl_override,
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client bl.spamcop.net,

After any change we make in Postfix, we need to reload to apply our changes:

service postfix reload


Adding SPF (Sender Policy Framework) Checking

Next we will add SPF (Sender Policy Framework) checking to Postfix: [7]

apt-get install postfix-policyd-spf-python

Back to editing main.cf:

nano /etc/postfix/main.cf

Add this line:

policy-spf_time_limit = 3600s

Now add check_policy_service unix:private/policy-spf to smtpd_recipient_restrictions just before the whitelist line we just added:

smtpd_recipient_restrictions =
  check_policy_service unix:private/policy-spf,
  check_client_access hash:/etc/postfix/rbl_override,
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client bl.spamcop.net,

Next we edit /etc/postfix/master.cf:

nano /etc/postfix/master.cf

Add this section to the bottom of the file:

policy-spf  unix  -       n       n       -       -       spawn
  user=nobody argv=/usr/bin/policyd-spf

And reload Postfix:

service postfix reload

We can test to make sure SPF checking is working by using the Wormly SMTP Server Test. Try sending a test message to your server, but use example@example.com as the sender email. You should see your server reject the message as Wormly is not authorized to send email messages on behalf of example.com.

You should make sure you have a valid SPF record for your domain in your DNS server. If you are just using your new Digital Ocean VPS, your SPF record could be as simple as "v=spf1 mx ~all". This will help prevent spammers from spoofing messages using your domain name.  For more information on creating a SPF record, visit SPF Wizard.  For more information on SPF in general, visit OpenSPF.


[Optional] Greylisting with postgrey

We already have blacklists and whitelists, why not add one more list into the mix?

A greylist works by delaying mail from untrusted servers.  If the sending mail server is standards-compliant, it will try resending the message, and then the server will accept it. Spam mailers do not typically try resending a message, and so the spam is blocked. Servers that do try resending mail will be added to postgrey’s white list, and will not be delayed in future. In other words, the first message from any given sender will be delayed, but subsequent ones will not.

This approach may not be appropriate for everyone. If you need to receive messages in a timely manner, you may not want to install postgrey.  I have come across certain servers that only retry sending messages every four hours.  Servers you have added to your own whitelist listed above will not be greylisted.

First we need to install postgrey: [8]

apt-get install postgrey

The default wait time for postgrey is 5 minutes, lets bring that down to one:

nano /etc/default/postgrey

Add --delay=60 to the end of POSTGREY_OPTS=:

POSTGREY_OPTS="--inet=10023 --delay=60"

Now to add postgrey to Postfix, we edit main.cf:

nano /etc/postfix/main.cf

Add the line check_policy_service inet: to smtpd_recipient_restrictions just before permit:

smtpd_recipient_restrictions =
  check_policy_service unix:private/policy-spf,
  check_client_access hash:/etc/postfix/rbl_override,
  reject_rbl_client zen.spamhaus.org,
  reject_rbl_client bl.spamcop.net,
  check_policy_service inet:,

Now restart both postgrey and Postfix:

service postgrey restart
service postfix restart

Head over to the Wormly SMTP Test once again and try to send a test message to your server. The message should fail. Checking the reason on the Wormly page, you should see Greylisted as the reason.  If you wait one or two minutes and then try again, you’ll see that the message will go through.

You can also check your logs:

tail /var/log/mail.log
Jul 10 13:23:53 hostname postgrey[4656]: action=greylist, reason=new, client_name=mail.example.com, client_address=xxx.xx.xxx.xx, sender=user@example.com, recipient=user@example.com
Jul 10 13:24:30 hostname postgrey[4656]: action=greylist, reason=early-retry (23s missing), client_name=mail.example.com, client_address=xxx.xx.xxx.xx, sender=user@example.com, recipient=user@example.com
Jul 10 13:24:58 hostname postgrey[4656]: action=pass, reason=triplet found, delay=65, client_name=mail.example.com, client_address=xxx.xx.xxx.xx, sender=user@example.com, recipient=user@example.com


And that should do it.  Using this configuration, I receive very little spam in my Junk E-Mail folder.  Those that do manage to make their way through, I promptly report to SpamCop and they usually stop rather quickly.

In my next tutorial we will be adding SSL/TLS protection to the webmail interface, Postfix and the POP/IMAP interfaces.  We will also be adding DKIM signing to outgoing messages.

As always, if you notice any errors, omissions or have anything to add that would make this tutorial better, please let me know in the comments below.

1. https://www.digitalocean.com/community/tutorials/how-to-install-and-setup-spamassassin-on-ubuntu-12-04
2. http://spamassassin.apache.org/full/3.1.x/doc/Mail_SpamAssassin_Conf.html#item_skip_rbl_checks
3. http://www.howtoforge.com/hardening-postfix-for-ispconfig-3
4. http://www.postfix.org/postconf.5.html#disable_vrfy_command
5. http://www.e-rave.nl/rate-limit-incoming-mail-on-postfix
6. http://www.howtoforge.com/how-to-whitelist-hosts-ip-addresses-in-postfix
7. https://help.ubuntu.com/community/Postfix/SPF
8. https://help.ubuntu.com/community/PostfixGreylisting

Self Hosted Gmail/Exchange Replacement

Awhile ago I was searching for a suitable Gmail replacement. I tried all of the usual suspects, and while they worked, some would have some features, other would have others, but nobody really had everything i was looking for, so I always ended sticking with Gmail.

My list really isn’t too demanding as to what I wanted:

  • Email using my own domain name
  • Contacts
  • Calendar
  • Mobile access with push

Also important, but not a deal breaker:

  • Tasks & Notes
  • A decent looking web interface

It was surprising how many email providers do not offer push email on the iPhone.  Android supports IMAP IDLE which offers push notifications, but if you want push on iOS, you need Exchange ActiveSync.

I did find a service called Zoho which actually looks promising.  Back then they didn’t offer very many users on their free account, but they have since increased that to 10 users on their free plan.   If you’re looking for a hosted Gmail alternative using your own domain, I’d suggest having a look at them.

After all of my reading, I came across exactly what I was looking for: Zarafa Community Edition.  It seemed to offer everything that I was looking for, with one catch;  I needed to host it myself.  Seeing as I was tired of 3rd party companies spying on my email, I thought hosting my own email server seemed like a logical next step.

Zarafa Community Edition offers IMAP & POP3 in both encrypted and unencrypted flavours as well as an iCal/CalDAV gateway.  The Zarafa WebApp (webmail interface) is fast, decent looking & surprisingly full featured.  Using Z-Push you can also use ActiveSync on your mobile devices!  (You can test drive the Zarafa Webapp here)

After a quick call to my ISP, I quickly realized that I wouldn’t be able to host an email server in my own home.  They block all of the necessary ports on residential internet connections to operate an email server.  (It is also apparently against their terms of service to operate servers on residential accounts, who knew?) – They were happy to try and sell me a business account for my home instead, which was more than twice the price, but I politely declined.

Ok, so I want to self host Zarafa, but I can’t host it myself, now what?  I need to find an affordable VPS (Virtual Private Server) provider.  Enter Digital Ocean!

Digital Ocean offers virtual servers which are powerful enough to host your own email server for only $5 per month.  If there is a catch, I haven’t found one yet and I’ve been using them for about a year.

Ok, enough back story, on to the good stuff;  how do you install Zarafa on a Digital Ocean VPS?

I’ll assume you have a basic understanding working with Linux at the command line.  I will go over the general packages you need to install, but you will need to configure your specific server to meet your needs.

First, create your Digital Ocean droplet using Ubuntu 14.04 64bit.  When creating your droplet you will be asked what you would like your hostname to be.  To simplify setting up encryption later on, I suggest you use only your domain name as your hostname.  You will be emailed your IP address, and root password.

While you are creating your droplet, you’ll need to think about the DNS settings for your domain name.  Digital Ocean offers DNS servers you can use with your server, or you can use the DNS servers you are already using.  Either way, you will need to make/change your MX record and point your domain and/or subdomain to your new IP address to use the webmail interface.

You will first need to install a couple of prerequisites.  We need a web server for webmail, along with PHP for scripting.  We also need a database to store the email, in our case MySQL.  This is also known as LAMP Stack (Linux, Apache, MySQL, PHP).

apt-get update
apt-get install apache2 php5 php5-mcrypt libapache2-mod-php5
apt-get install mysql-server libapache2-mod-auth-mysql php5-mysql

You’ll be asked while MySQL is installing to create a root SQL password.  Remember this password as we’ll be using it later to both secure MySQL and to create a Zarafa database.

We also need to secure this MySQL installation:


Now lets create the database which Zarafa will use: (remember to replace some_password with an actual password)

mysql -u root -p
GRANT ALL ON zarafa.* TO 'zarafa'@'localhost' IDENTIFIED BY 'some_password';

Remember this database password, we’ll need it later when installing Zarafa.

We also need to install the mail transfer agent portion of the server, we’ll be using Postfix:

apt-get install postfix

Follow the directions here setting up Postfix, making sure that your system mail name matches the hostname you configured in Digital Ocean.  It is very important for a mail server that the reverse DNS name matches the name presented by the mail server.

We’re now ready to download and install Zarafa: (replace file name with most recent version)

cd /tmp
wget http://download.zarafa.com/community/final/7.1/7.1.10-44973/zcp-7.1.10-44973-ubuntu-14.04-x86_64-free.tar.gz
tar -xzf zcp-7.1.10-44973-ubuntu-14.04-x86_64-free.tar.gz
cd zcp-7.1.10-44973-ubuntu-14.04-x86_64

Zarafa will install a couple of additional packages, as well as the servers. I just selected the defaults for all options, as I want to use all of the services Zarafa offers.  You’ll also be asked for the database information we created earlier.   If all goes well, Zarafa will install and you’ll be asked if you want to start all of the services.

We now need to configure postfix to pass received messages to Zarafa: [1]

postconf -e 'mailbox_command = /usr/bin/zarafa-dagent "$USER"'
service postfix reload

By default Zarafa disables POP and IMAP access for all new users. We need to enable IMAP for each user as this is what the SASL daemon uses to autenticate:

nano /etc/zarafa/server.cfg

Find the line disabled_features:

disabled_features = imap pop3

And remove imap: (If you also plan on allowing POP3 access for your users, you can remove pop3 as well.)

disabled_features = pop3

Reload the zarafa server to apply the changes:

service zarafa-server reload

Next we need to setup SASL authentication in Postfix to use your Zarafa usernames and passwords: [2]

apt-get install libsasl2-2 sasl2-bin libsasl2-modules
mkdir /etc/postfix/sasl
nano /etc/postfix/sasl/smtpd.conf

Add these two lines to smtpd.conf:

pwcheck_method: saslauthd
mech_list: plain login

Next we edit another file:

nano /etc/default/saslauthd

Edit the existing lines of your file to match these, adding or uncommenting as necessary: [3]

DESC="SASL Authentication Daemon"
OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -r"

Now we tell postfix to use these new settings:

postconf -e 'smtpd_sasl_auth_enable = yes'
postconf -e 'smtpd_sasl_path = smtpd'
dpkg-statoverride --force --update --add root sasl 755 /var/spool/postfix/var/run/saslauthd

The last command may report an error that “–update given” and the “/var/spool/postfix/var/run/saslauthd” directory does not exist. You can ignore this because when you start saslauthd next it will be created.

Now we can start the new service:

service saslauthd start

Now we’ll need to create your first user. Zarafa uses a command line utility to add, delete & modify users:

zarafa-admin -c <username> -p <some_password> -f 'Full Name' -e <email@address.com>

We also need to let the system know we’ve created this user, so postfix also knows about it:

useradd -s /bin/false <username>

Your system is now ready to send and receive email.  Now we can activate the webmail interface:

ln -s /etc/apache2/sites-available/zarafa-webapp /etc/apache2/sites-enabled/zarafa-webapp.conf
service apache2 reload

Now you can browse to http://<your droplet name or IP>/webapp and login using the username and password you just created.

A great tool to test your new email server is the Wormly SMTP Test.  Go to that site and enter your email server’s name or IP, a from email address and your newly created email address in the To field.  If everything was installed correctly you should see a test message appear in your webmail interface’s Inbox.  From this point, you should also be able to send a test message out.

Now that we can send and receive email, lets setup ActiveSync (push) for mobile devices.  To do this, we’ll use Z-Push: (replace file name with most recent version)

cd /tmp
wget http://download.z-push.org/final/2.1/z-push-2.1.3-1892.tar.gz
tar -xzf z-push-2.1.3-1892.tar.gz
mv ./z-push-2.1.3-1892 /usr/share/z-push
mkdir /var/lib/z-push /var/log/z-push
chmod 755 /var/lib/z-push /var/log/z-push
chown -R www-data:www-data /var/lib/z-push /var/log/z-push /usr/share/z-push

Z-Push is now installed, we just need to configure it and add it to Apache. We’ll will need to edit a couple of configuration files:

nano /usr/share/z-push/config.php

Scroll down and replace define('TIMEZONE', ''); with define('TIMEZONE', 'America/Toronto'); or whatever your local timezone is.  Next you will need to replace define('BACKEND_PROVIDER', ''); with define('BACKEND_PROVIDER', 'backendZarafa'); That’s all we need to configure in that file, Press CTRL-X to exit and Y to save your changes.

Next we need to add Z-Push to Apache:

nano /etc/apache2/sites-enabled/000-default.conf

Add these lines to the end of the file, just before </VirtualHost>:

Alias /Microsoft-Server-ActiveSync /usr/share/z-push/index.php
<Directory /usr/share/z-push>
php_flag magic_quotes_gpc off
php_flag register_globals off
php_flag magic_quotes_runtime off
php_flag short_open_tag on

Now we can reload Apache:

service apache2 reload

And now we have ActiveSync setup!  To test to make sure it’s working, go to http://<your droplet name or IP>/Microsoft-Server-ActiveSync.  You should be asked for a username and password.  Enter the credentials you used when creating your account in Zarafa.  You should then get a message saying “GET not supported”.  This is good.  Now give it a try on your mobile device.

You should be able to create calendar entries, notes, tasks, contacts and of course email on your mobile device and everything should be almost instantaneously synced on your server and viewable via the webmail interface.

That pretty much does it!  Provided you correctly configured your domain’s MX record, you should be able to send and receive email using your new mail server, no matter whether it’s through webmail, mobile or using a client like Thunderbird (via IMAP & SMTP).

The only problem now is, there is no anti-spam or encryption installed.  My next two posts will deal with adding these important components, making your new installation actually usable.

If you notice any errors, omissions or have anything to add that would make this tutorial better, please let me know in the comments below.

1. http://www.zarafa.com/wiki/index.php/Installing_Zarafa_from_Ubuntu_Repository
2. https://help.ubuntu.com/community/Postfix
3. http://www.zarafa.com/wiki/index.php/SMTP-Auth_for_IMAP_users