Web server setup

Server setup
← Previous	Next →
PostgreSQL config	Mediawiki farm

I have Apache2 running on both the server and desktop machines. The webserver on the server serves public pages, while the one on the desktop is for pages private to my LAN. The server webserver hosts a number of virtual sites. Most of them are various Mediawiki instances. For those things that don't fit within in a Mediawiki, there is also a basic HTML site set up. Finally, I have a webmail interface to my mail server, running over a SSL connection. Mediawiki and webmail setup are discussed on other pages; here I'll just describe the basic HTML setup, including allowing a secure connection.

Getting Apache2 running

Apache2 should already be running as part of the LAMP stack installed when the OS was installed. If not, add it with

root@server:~# apt-get install apache2-mpm-worker

Name-based virtual hosts

Name-based virtual hosts allow one webserver, with one IP number, to server multiple websites depending on the server name used to request the page.

Enable name-based virtual hosts by modifying the end of /etc/apache2/ports.conf to include the two NameVirtualHost lines:

NameVirtualHost *:80
NameVirtualHost *:443

There will be one virtual host for each site. The files for each site will reside in a separate directory under /var/www and each site will have a separate configuration file in /etc/apache2/sites-available. A typical configuration file is below:

<VirtualHost *:80>
       ServerAdmin webmaster@localhost

       DocumentRoot /var/www/site.domain.tld
       ServerName site.domain.tld

       <Directory />
               Options FollowSymLinks
               AllowOverride None
       </Directory>
       <Directory /var/www/>
               Options Indexes FollowSymLinks MultiViews
               AllowOverride None
               Order allow,deny
               Allow from all
       </Directory>

       ScriptAlias /cgi-bin/ /var/www/cgi.site.domain.tld/
       <Directory "/var/www/cgi.site.domain.tld">
               AllowOverride None
               Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
               Order allow,deny
               Allow from all
       </Directory>

       ErrorLog /var/log/apache2/error.log

       LogLevel warn
       CustomLog /var/log/apache2/site.domain.tld.access.log combined

       ServerSignature Off

       Alias /doc/ "/usr/share/doc/"
       <Directory "/usr/share/doc/">
               Options Indexes MultiViews FollowSymLinks
               AllowOverride None
               Order deny,allow
               Deny from all
               Allow from 127.0.0.0/255.0.0.0 192.168.1.0/255.255.255.0 ::1/128
       </Directory>
</VirtualHost>

Repeat this file for each virtual host you want to set up. Change the ServerName, DocumentRoot, and CustomLog settings to reflect the site's name and location of files. Also note the non-default location of the cgi-bin directory. This directory is outside the document root of any website (to prevent its contents being viewed directly via Apache), is owned by root, and permissions of 755. Also note that the contents of the /usr/share/doc/ directory are only available to users on the LAN.

When a site is ready, enable it with the command:

root@server:~# a2ensite /etc/apache2/sites-available/site-settings-file

then reload the configuration:

root@server:~# systemctl reload apache2.service

Setting a default virtual host

When there are several virtual hosts with servernames (or aliases) that could match a request, Apache2 uses the first one it comes across. That means the server defined in the configuration file earliest in lexicographical order. Therefore, make sure that the virtual host you want as the default is defined in a file with a 00 prefix, such as /etc/apache2/site-available/00-default.domain.tld.

You'll probably also want to disable the default host that comes with Apache2.

root@server:~# a2dissite default
root@server:~# a2ensite 00-default.domain.tld
root@server:~# ssystemctl reload apache2.service

Stopping version number leakage

By default, Apache reveals the full version number of itself, the OS, and all modules attached, whenever there's an error. This can make life easier for someone wanting to hack your system. You can prevent Apache revealing all this information with a couple of settings.

In /etc/apache2/apache2.conf, set the following directives:

ServerTokens Prod
ServerSignature Off

In each virtual host configuration file, set:

ServerSignature Off

(it's already set in the sample above) Finally, reload the configuration:

root@server:~# systemctl reload apache2.service

Secure HTTP

I use the Secure HTTP server for my webmail You'll need to look at the Webmail setup page for the rest of the configuration.

The first step is to enable the Apache rewriting engine and the SSL module:

root@server:~# a2enmod ssl

Now, create the configuration file for the squirrelmail.domain.tld site, /etc/apache2/sites-available/squirrelmail.domain.tld

<VirtualHost *:80>
       ServerAdmin webmaster@localhost

       ServerName squirrelmail.domain.tld

       Redirect permanent / https://squirrelmail.domain.tld/

       CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined
       ServerSignature Off
</VirtualHost>

<VirtualHost *:443>
       ServerAdmin webmaster@localhost

       SSLEngine On
       SSLCertificateChainFile /etc/letsencrypt/live/webmail.domain.tld/chain.pem
       SSLCertificateKeyFile /etc/letsencrypt/live/webmail.domain.tld/privkey.pem
       SSLCertificateFile /etc/letsencrypt/live/webmail.domain.tld/cert.pem

       DocumentRoot /var/www/www.domain.tld
       ServerName squirrelmail.domain.tld

       <Directory />
               Options FollowSymLinks
               AllowOverride None
       </Directory>

       ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
       <Directory "/usr/lib/cgi-bin">
               AllowOverride None
               Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
               Order allow,deny
               Allow from all
       </Directory>

       ErrorLog /var/log/apache2/error.log

       # Possible values include: debug, info, notice, warn, error, crit,
       # alert, emerg.
       LogLevel warn

       CustomLog /var/log/apache2/access.squirrelmail.domain.tld.log combined 
       ServerSignature Off
</VirtualHost>

Note that this file contains the settings for both a secure and an insecure site. The insecure site, on port 80, causes an immediate redirections to the secure site on port 443.

Finally, ask Apache to listen to port 443. Add this line to /etc/apache2/ports.conf

Listen 443

Enable the site:

root@server:~# a2ensite /etc/apache2/sites-available/squirrelmail.domain.tld

then restart the server:

root@server:~# systemctl restart apache2.service

Restart Apache and you should be able to see the secure site. It should show the same content as the base site, www.domain.tld. We'll do Webmail setup later.

Note to self

On Apache version 2.4.8 and above, replace:

SSLCertificateChainFile /etc/letsencrypt/live/domain.tld/chain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem
SSLCertificateFile /etc/letsencrypt/live/domain.tld/cert.pem

with:

SSLCertificateKeyFile /etc/letsencrypt/live/domain.tld/privkey.pem
SSLCertificateFile /etc/letsencrypt/live/domain.tld/fullchain.pem

Fix security holes

Modify /etc/apache2/conf-enabled/security.conf to turn off server signatures:

ServerSignature Off

Add modules needed

root@server:~# a2enmod headers
root@server:~# a2enmod cgi

I use mod_evasive to prevent ill-behaved bots overloading the server.

root@server:~# apt install libapache2-mod-evasive
root@server:~# a2enmod evasive

The default settings in /etc/apache2/mods-enabled/evasive.conf seem reasonable, but I uncomment and set DOSEmailNotify.

Update /etc/apache2/mods-enabled/deflate.conf to include text/javascript in the first line

<IfModule mod_deflate.c>
   <IfModule mod_filter.c>
       AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript
       AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
       AddOutputFilterByType DEFLATE application/rss+xml
       AddOutputFilterByType DEFLATE application/xml
   </IfModule>
</IfModule>

Restart the server to incorporate the changes.

root@server:~# systemctl restart apache2.service

Install PHP

I install PHP in a moment for the Mediawiki farm setup, but it needs tweaking first to allow HTTP2 to be enabled.

Install PHP and the FastCGI extension

root@server:~# aptitude install php php-fpm php-apcu libapache2-mod-fastcgi

Enable PHP and the FastCGI modules

root@server:~# a2enmod php7.2
root@server:~# a2enmod php7.2-fpm
root@server:~# a2dismod php7.0
root@server:~# systemctl restart apache2

Enable HTTP2

To enable HTTP2, switch PHP to use the FPM module and Apache to the event handler.

root@server:~# a2enmod proxy_fcgi
root@server:~# a2enconf php7.2-fpm
root@server:~# a2dismod mpm_prefork
root@server:~# a2enmod mpm_event
root@server:~# systemctl restart apache2

Test the PHP installation by creating a file /var/www/site.domain.tld/test.php containing

<?php phpinfo(); ?>

then visit that file in a browser. It should say

Server API FPM/FastCGI

near the top.

Then enable HTTP2 by creating the file /etc/apache2/conf-available/http2.conf

Protocols h2 http/1.1

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

Then enable the module.

root@server:~# a2enconf http2
root@server:~# a2enmod http2
root@server:~# systemctl restart apache2

Test it by pointing the KeyCDN tester at the domain.

Once the FCGI plugin is working, allow it to use more workers by modifying /etc/php/7.2/fpm/pool.d/www.conf to update these lines (including commenting out pm.start_servers:

pm.max_children = 5
; pm.start_servers = 2
pm.min_spare_servers = 2
pm.max_spare_servers = 6

Then restart the FCGI server.

root@server:~# systemctl restart php7.2-fpm.service

The server should now be using all the CPU while serving many concurrent requests.

Prevent bot scraping

The standard robots.txt file should prevent most bots abusing the web server, but not all bots respect it. In particular, the web-scraping bots of AI companies can consume huge amounts of bandwidth. The AI robots repository has details on various ways to stop these bots.

I ended up blocking them in with an additional configuration file that returns a 403 (Forbidden) code if any of the AI bots requests a page. This is based on the ai.robots.txt Nginx configuration file.

Create the file /etc/apache2/conf-available/ai-bot-blocklist.conf:

 RewriteEngine on
 RewriteCond %{HTTP_REFERER} AddSearchBot|AI2Bot| ... |YouBot [NC]
 RewriteRule . - [R=403,L]

(include the full bot list from the AI robots repository.)

Enable the configuration and reload Apache.

root@server:~# a2enmod ai-bot-blocklist
root@server:~# systemctl reload apache2