Set up an Email Server using Postfix, Dovecot and MySQL in Ubuntu 14.04 – Part 1

This tutorial will guide you in setting up your own email server using Postfix, Dovecot and MySQL along with posty REST API for administration. In this tutorial we will configure it to use IMAP. However you can change the configurations to use POP3.

1. Overview Terms:

  • SMTP – Simple Mail Transfer Protocol
  • IMAP – Internet Message Access Protocol
  • IMAPS – IMAP Secure
  • POP3 – Post Office Protocol 3
  • POP3S – POP3 Secure
  • MTA – Mail Transfer Agent
  • LMTP – Local Mail Transfer Protocol

The following diagram shows a simple overview of how an email works:

email_overview

  • The sender sends an email to user@example.com using SMTP
  • The DNS Server finds the MX records for example.com and returns the corresponding server address to route the email
  • Postfix receives the mail on the receiving mail server
  • Postfix validates the address of the recipient by checking if it exists in the database and sends it to Dovecot
  • Dovecot LMTP processes the incoming mail
  • Dovecot apply filters and finally saves it to the corresponding mailbox

2. Ports

Before we begin with the installation, you might first want to check if all the required ports are open

  • SMTP
    • 25 and 465
    • 587 with SSL for outgoing mail
  • IMAP
    • 143
    • 993 with SSL

Checking firewall:

Use the following command to see which ports are listening:

sudo netstat -tulpn

You can also use nmap to check if any port is filtered in your network:

nmap youripaddress

3. Host Confguration

Edit hosts file:

vi /etc/hosts

127.0.0.1 localhost 127.0.1.1 yourserverhostname.maildomain.tld yourserverhostname.

Make sure to replace ‘yourserverhostname’ and ‘maildomain.tld’ with yours.

You can use ‘hostname’ command to check server hostname

For E.g:  127.0.0.1 localhost 127.0.1.1 email-server.mailserver.com email-server

4. SSL Certificates

For secure connections we need to set up the ssl certificates. To obtain one you can you use any of the CA services by any trusted Issuer.

Note: Before you choose an issuer make sure it is trusted all major browsers.

To generate a new certificate use the following command:

openssl req -newkey rsa:2048 -keyout mailserver.key -out mailserver.csr

Submit the csr file and get it signed by your cartificate authority. After receiving the signed certificate, you should have the following:

  1. root.pem
  2. intermediate_ca.pem
  3. mailserver.pem

Copy the certificate file to /etc/ssl/certs/mailserver.pem and key file to /etc/ssl/private/mailserver.key

5. Software required to install:

  • MySQL (version 5.6)For storing email users, domain and aliases.
  • Dovecot (version 2.2.9)An open source IMAP and POP3 email server for Linux/UNIX-like systems which also includes an LMTP server.
  • Postfix: (version 2.11.0)An open source MTA
  • PhpMyAdmin (Optional):For managing mysql databases
  • Roundcube:An open source webmail client.
  • Posty Rest APIhttp://www.posty-soft.org/

    Open source administration utility for mail servers based on postfix and dovecot.

6. Installation

Now we will install the required software packages:

  • Zip Utilities:

    sudo apt-get install zip unzip
  • MySQL Server:

    To install mysql use the following command:

    sudo apt-get install mysql-server
  • Postfix:

    During installation, you will be asked to configure some settings in the ‘Postfix Configuration’ dialog.

    Make sure you use the following settings: For ‘General type of mail configuration’ choose ‘internet site’ option

    For ‘System mail name’ enter

    yourserverhost.maildomain.tld, replacing ‘yourserverhost’ and ‘maildomain.tld’ with yours

    E.g: email-server.mailserver.com

    To install postfix use the following command:

    sudo apt-get install postfix postfix-mysql
  • Remove exim, the default mail server using the command below:
    sudo apt-get --purge remove 'exim4*'
  • Dovecot:

    Install dovecot using the following command:

    sudo apt-get install dovecot-mysql dovecot-imapd dovecot-managesieved dovecot-lmtpd
  • Roundcube:

    Install roundcube using the following command:

    sudo apt-get install roundcube roundcube-plugins
  • PhpMyAdmin:

    To install phpmyadmin use the command:

    sudo apt-get install phpmyadmin
  • Posty API:

    To install posty API, we need to install ruby on rails. Download the zip file and unzip the folder using the commands below:

    wget htps://github.com/posty/posty_api/archive/master.zip
    mv master.zip posty_api.zip
    unzip posty_api.zip
    • Set up ruby on rails:

      This will be used to set up the REST Server. For more information you can check it here. Use the following commands to set up ruby on rails:

      sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev nodejs
      sudo apt-get install libgdbm-dev libncurses5-dev automake libtool bison libffi-dev
      gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
      curl -sSL https://get.rvm.io | bash -s stable
      source /etc/profile.d/rvm.sh
      rvm install 2.3.0
      rvm use 2.3.0 --default
    • Install bundler

      gem install bundler

7. Configuration

  • MySQL:

    Create database for storing email credentials.

    
          CREATE DATABASE emailserver;
    
          

    Create mysql user for using the emailserver database.

    
          GRANT ALL ON emailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mailuserpass';
    
          
  • Apache Server:

    We need to configure SSL certificates for using secure http. For this edit the file /etc/apache2/sites-available/default-ssl.conf and change the lines below:

          SSLCertificateFile    /etc/ssl/certs/mailserver.pem
          SSLCertificateKeyFile /etc/ssl/private/mailserver.key
          

    Enable SSL encryption:

    a2enmod ssl

    Enable https:

    a2ensite default-ssl

    Restart the server:

    service apache2 reload

    Load your web page at https://youripaddress

  • PhpMyAdmin:

    Edit /etc/apache2/sites-available/default-ssl.conf include the following:

    
          Include /etc/phpmyadmin/apache.conf
    
          

    Reload apache

    service apache2 reload
  • Posty API:

    Change path to posty_api

    Before we proceed, we need to install mysqllib development library and then the required bundle:

    apt-get install libmysqlclient-dev
    bundle install --with mysql

    We will use schema generated by posty. For this we nned to copy config/database.mysql2.yml to config/database.yml and change mysql conncetion parameters.

    Create the tables using the following command:

    rake db:migrate

    This should generate the tables as follows:

          api_keys domain_aliases_view
          schema_migrations
          user_aliases_view
          users_view
          virtual_domain_aliases
          virtual_domains
          virtual_transports
          virtual_user_aliases
          virtual_users
          

    Generate api keys:

    rake api_key:generate

    This will return your api auth_token to be used when using the REST API. E.g

          d5a83739c8ade22bcb1f704849f50d6a
          

    This will be used to authenticate request using the REST
    API

    Start the rails app server:

    rackup

    Run as a daemon process:

    rackup -D

    Okay, lets test by creating some email accounts, virtual domains and aliases. These will be used later when configuring postfix in order to tell postfix to validate email accounts.

    Using the token generated above create a test virtual domain. Remeber to replace the ip address with yours.

          http://youripaddress:9292/api/v1/domains?auth_token=d5a83739c8ade22bcb1f704849f50d6a
          {"name": "mailserverdemo.com"}
          

    Create a test user

          http://youripaddress:9292/api/v1/domains/mailserverdemo.com/users?auth_token=d5a83739c8ade22bcb1f704849f50d6a
          {"name": "test", "password": "test", "quota": 250000}
          

    Create a domain alias

          http://youripaddress:9292/api/v1/domains/mailserverdemo.com/aliases?auth_token=d5a83739c8ade22bcb1f704849f50d6a
          {"name": "mailserverdemoalias.com"}
          

    Create a user alias

          http://youripaddress:9292/api/v1/domains/mailserverdemo.com/users/lamteiwahlang/aliases?auth_token=d5a83739c8ade22bcb1f704849f50d6a
          {"name": "demo"}
          
  • Postfix:

    Postfix has two main configuration files, located in

    /etc/postfix/main.cf and

    /etc/postfix/master.cf

    You can read more about the settings in http://www.postfix.org/BASIC_CONFIGURATION_README.html

    In order for Postfix to work with SQL, we need to create configuration files to mapping each setting to a file.

    virtual_mailbox_domains virtual_mailbox_maps virtual_alias_domains virtual_alias_maps

    1. Virtual Mailbox Domains (virtual_mailbox_domains)

      Create a new configuration for virtual mail box domain using mysql in /etc/postfix/mysql-virtual-mailbox-domains.cf

                user = mailuser
                password = mailuserpass
                hosts = 127.0.0.1
                dbname = emailserver
                query = SELECT 1 FROM virtual_domains WHERE name='%s'
                

      Add mapping for virtual_mailbox_domains to main.cf using the command below:

      postconf virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

      Test if the mapping works:

      postmap -q mailserverdemo.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

      Should return 1 if mapping was successful

    2. Virtual User Mailbox Maps (virtual_mailbox_maps)

      vi /etc/postfix/mysql-virtual-mailbox-maps.cf

                user = mailuser
                password = mailuserpass
                hosts = 127.0.0.1
                dbname = emailserver
                query = SELECT 1 FROM users_view WHERE email='%s'
                

      Add mapping for virtual mailbox to main.cf using the command below:

      postconf virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

      Test the mapping:

      postmap -q test@mailserverdemo.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

      Should return 1 if mapping was successful

    3. Virtual Domain Alias Maps (virtual_alias_domains)

      vi /etc/postfix/mysql-virtual-domain-aliases.cf

                user = mailuser
                password = mailuserpass
                hosts = 127.0.0.1
                dbname = emailserver
                query = SELECT destination FROM domain_aliases_view WHERE source='%s'
                

      Add mapping for virtual_alias_domains to main.cf using the command below:

      postconf virtual_alias_domains=mysql:/etc/postfix/mysql-virtual-domain-aliases.cf

      Test the mapping:

      postmap -q @mailserverdemoalias.com mysql:/etc/postfix/mysql-virtual-domain-aliases.cf

      Should return @mailserverdemo.com if mapping was successful

    4. Virtual User Alias Map (virtual_alias_maps)

      vi /etc/postfix/mysql-virtual-user-aliases.cf

                user = mailuser
                password = mailuserpass
                hosts = 127.0.0.1
                dbname = emailserver
                query = SELECT destination FROM user_aliases_view WHERE source='%s'
                
      postconf virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-user-aliases.cf

      Add mapping for virtual_alias_maps to main.cf using the command below:

      postmap -q demo@mailserverdemo.com mysql:/etc/postfix/mysql-virtual-user-aliases.cf

      Should return test@mailserverdemo.com

    5. Catch-all aliases

      To forward all email addresses in a domain to an email address:

      vi /etc/postfix/mysql-email2email.cf

                user = mailuser
                password = mailuserpass
                hosts = 127.0.0.1
                dbname = emailserver
                query = SELECT email FROM users_view WHERE email='%s'
                
      postmap -q test@mailserverdemo.com mysql:/etc/postfix/mysql-email2email.cf

      test@mailserverdemo.com

      Map both catch-all and virtual aliases for a user:

      postconf virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-user-aliases.cf, mysql:/etc/postfix/mysql-email2email.cf
    6. Configure Postfix to use Dovecot’s LMTP for email delivery:

      postconf virtual_transport=lmtp:unix:private/dovecot-lmtp
    7. Make Postfix use Dovecot for authentication:

      Enables TLS for authentication.

      postconf smtpd_sasl_type=dovecot
      postconf smtpd_sasl_path=private/auth
      postconf smtpd_sasl_auth_enable=yes
      postconf smtpd_tls_security_level=may
      postconf smtpd_tls_auth_only=yes
      postconf smtpd_tls_cert_file=/etc/ssl/certs/mailserver.pem
      postconf smtpd_tls_key_file=/etc/ssl/private/mailserver.key
      postconf 'smtpd_tls_mandatory_protocols=!SSLv2,!SSLv3'

      The last one is ensure that SSL2 and SSL3 are not used in order to prevent Poodle attack.

    8. Set permissions for Postfix:

      chgrp postfix /etc/postfix/mysql-*.cf
      chmod 644 /etc/postfix/mysql-*.cf
  • Dovecot:

    Create a new system user that will own all virtual mailboxes

    groupadd -g 5000 vmail
    useradd -g vmail -u 5000 vmail -d /var/vmail -m
    chown -R vmail.vmail /var/vmail

    Dovecot configurations can be found in /etc/dovecot/ and /etc/dovecot/conf.d directries.

    1. 10-auth.conf

      Tell dovecot to use MySQL database backend for authentication So uncomment !include auth-sql.conf.ext

                #!include auth-system.conf.ext
                !include auth-sql.conf.ext
                #!include auth-ldap.conf.ext 

      Edit auth-sql.conf.ext and change the following:

      Password database lookups

      We need to tell dovecot to use the SQL connection in dovecot-sql.conf.ext to look up for the password which will return a field named “password”

                passdb {
                    driver = sql args = /etc/dovecot/dovecot-sql.conf.ext
                }
                

      We can choose to use static UID and GID with a looking template for userdb lookup, this will be more faster since we don’t need to use SQL quries.

                userdb {
                    driver = static args = uid=vmail gid=vmail home=/var/vmail/%d/%n
                }
                
    2. 10-mail.conf

      Change the mail_location setting to:

                mail_location = maildir:/var/vmail/%d/%n/Maildir
                

      This is the directory for storing emails. Here %d will be the name of the domain, and %n the name of the user.

      Change the separator

                separator = .
                

      Here we use . as separator for backward compatibility

    3. 10-master.conf

      Change ‘service auth’ to make Postfix authenticate using Dovecot:

                # Postfix smtp-auth
                unix_listener /var/spool/postfix/private/auth {
                    mode = 0660 user = postfix group = postfix
                }
                

      This will allow communication with Postfix we tell Dovecot to use the communication socket file in that chroot area.

    4. 10-ssl.conf

      Point ssl certificate and key to their path

                ssl_cert = /etc/ssl/certs/mailserver.pem
                ssl_key = /etc/ssl/private/mailserver.key
                

      Enable TLS/SSL encryption by setting:

                ssl = yes
                
    5. 15-mailboxes.conf

                mailbox Junk {
                    auto = subscribe
                    special_use = \Junk
                }
                mailbox Trash {
                    auto = subscribe
                    special_use = \Trash
                }
                
    6. Tell dovecot to use mysql database using the connection parameters in /etc/dovecot/dovecot-sql.conf.ext

                driver = mysql
                connect = host=127.0.0.1
                dbname=emailserver
                user=mailuser
                password=mailuserpass
                default_pass_scheme = SHA512-CRYPT
                password_query = SELECT email as user, password FROM users_view WHERE email='%u';
                
    7. Set permissions for Dovecot:

      chown root:root /etc/dovecot/dovecot-sql.conf.ext
      chmod 644 /etc/dovecot/dovecot-sql.conf.ext
    8. Configure Dovecot to receive emails from Postfix:

      Edit Dovecot’s configuration file that deals with the LMTP daemon.

      You can find it at /etc/dovecot/conf.d/10-master.conf.In “service lmtp” section change it, so it looks like below:

                service lmtp {
                    unix_listener /var/spool/postfix/private/dovecot-lmtp {
                        group = postfix mode = 0600 user = postfix
                    }
                }
                

      Enable ssl and their ports in service imap-login

                inet_listener imap {
                    port = 143
                }
                inet_listener imaps {
                    port = 993
                    ssl = yes
                }
                

      Setting port to 0 will disable the protocol

    9. Enable mail plugins in 10-mail.conf:

      
                mail_plugins = $mail_plugins quota
      
                

      Edit the file /etc/dovecot/conf.d/20-lmtp.conf and within the “protocol lmtp” section change the “mail_plugins” line to:

      
                mail_plugins = $mail_plugins sieve
      
                

      Edit 20-imap.conf

      
                mail_plugins = $mail_plugins imap_quota
      
                

Next Steps: So far we have only installed and configured Postfix, Dovecot and MySQL to get them to work together. In the next part we will test both incoming and outgoing email delivery using ‘mail’ command and Roundcube. We will also see how to use posty API for administration.

Advertisements