Mediawiki farm setup

From Smith family
Jump to: navigation, search
Server setup
← Previous Next →
Web server Webmail

Many of the virtual web hosts on the server will run Mediawiki instances, with a separate Mediawiki for each virtual host. Setting them up isn't really any more complicated than setting up a single Mediawiki instance, but a little care is needed.

Install the packages

Debian (and hence Ubuntu) no longer supply packages of Mediawiki. Instead, get the Mediawiki LTS versions from the Mediawiki PPA

root@server:~# add-apt-repository ppa:legoktm/mediawiki-lts
root@server:~# aptitude update
root@server:~# aptitude install mediawiki libapache2-mod-php7.0 imagemagick php-apcu 
root@server:~# aptitude install php-gd php-json php-mysql php-xml pdo_mysql

Ubuntu doesn't seem to inlcude the apc PHP cache, which has a great effect on Mediawiki performance. Instead, enable the apcu cache:

root@server:~# cp /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php.original
root@server:~# cp /var/lib/mediawiki/includes/objectcache/ObjectCache.php /var/lib/mediawiki/includes/objectcache/ObjectCache.php.original
root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/libs/objectcache/APCBagOStuff.php
root@server:~# sed -i 's/apc_/apcu_/g' /var/lib/mediawiki/includes/objectcache/ObjectCache.php

Creating the farm

The basic idea is that each Mediawiki has a its own database and its own copy of all the Mediawiki configuration and files. We can cheat slightly by having each Mediawiki instance contain a link to a single set of program files, but all the configuration files are unique to each Mediawiki instance.

  • Create the file /var/www/mediawiki-symlinks to create all the symlinks for you; make the file executable.
#!/bin/bash

for f in /usr/share/mediawiki/api.php \
        /usr/share/mediawiki/autoload.php  \
        /var/lib/mediawiki/extensions/  \
        /usr/share/mediawiki/img_auth.php  \
        /usr/share/mediawiki/includes/  \
        /usr/share/mediawiki/index.php  \
        /usr/share/mediawiki/languages/  \
        /usr/share/mediawiki/load.php  \
        /usr/share/mediawiki/maintenance/  \
        /usr/share/mediawiki/mw-config  \
        /usr/share/mediawiki/opensearch_desc.php  \
        /usr/share/mediawiki/profileinfo.php  \
        /usr/share/mediawiki/resources  \
        /usr/share/mediawiki/serialized  \
        /usr/share/mediawiki/skins  \
        /usr/share/mediawiki/thumb_handler.php \
        /usr/share/mediawiki/thumb.php  \
        /usr/share/mediawiki/vendor  \
        /usr/share/mediawiki/wiki.phtml 
do
      ln -s $f
done

cp -pr /var/lib/mediawiki/cache .
cp -pr /var/lib/mediawiki/config .
cp -pr /var/lib/mediawiki/images .
  • For each wiki, create a directory where the wiki's files will sit, and add the copies:
root@server:~# mkdir -p /var/www/site.domain.tld/mediawiki
root@server:~# cd /var/www/site.domain.tld/mediawiki
root@server:~# /var/www/mediawiki-symlinks
  • Each site will also need it's own directory for creating temporary files. This need not be under the web root.
root@server:~# mkdir -p /var/www/site.domain.tld/images/temp
root@server:~# mkdir /var/www/site.domain.tld/images/thumb
root@server:~# chown -R www-data:www-data /var/www/site.domain.tld/images/temp
  • If you're updating or transferring a site from another host, set up the database and users first.
  • Modify the Apache configuration file for this site, /etc/apache2/sites-available/site.domain.tld:
<VirtualHost *:80>
       ServerAdmin webmaster@localhost

       DocumentRoot /var/www/site.domain.tld
       ServerName site.domain.tld
       ServerAlias domain.tld

       Alias /mediawiki /var/www/site.domain.tld/mediawiki
       SetEnv MW_INSTALL_PATH /var/www/site.domain.tld/mediawiki # Bugfix: Mediawiki v1.31.1 doesn't like wiki farms built with symlinks, so need to set the MW_INSTALL_PATH explicitly

       <Directory /var/www/site.domain.tld/mediawiki/>
               Options +FollowSymLinks
               AllowOverride All
               order allow,deny
               allow from all
       </Directory>

       # some directories must be protected
       <Directory /var/www/site.domain.tld/mediawiki/config>
               Options -FollowSymLinks
               AllowOverride None
       </Directory>
       
       <Directory /var/www/site.domain.tld/mediawiki/images>
               Options -FollowSymLinks
               AllowOverride None
       </Directory>

       # Rewrite rule will go here: see below 

       ErrorLog /var/log/apache2/error.log

       # Possible values include: debug, info, notice, warn, error, crit,
       # alert, emerg.
       LogLevel warn

       CustomLog /var/log/apache2/access.log combined
       ServerSignature On
</VirtualHost>
(The setting of MW_INSTALL_PATH is taken from the phabricator discussion.)
  • Reload the Apache config:
root@server:~# a2ensite site.domain.tld
root@server:~# systemctl reload apache2.service
  • Now got to http://site.domain.tld/mediawiki/ and follow the instructions to set up the wiki. During guided creation, ask for "Authorised editors only" to restrict editing and account creation, and enable uploads. Make sure that you give each wiki a distinct database name.
  • The setup process will finish by downloading a copy of the LocalSettings.php file. Copy it into into the mediawiki root at /var/www/domain.tld/mediawiki:
  • Add these lines to LocalSettings.php:
## The file path for temporary files. Was /tmp
$wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp";
Yes, this needs to be an absolute path on the webserver's filesystem.

That should be everything needed to get the wiki working.

Rewriting paths

Now to get rid of the /mediawiki/ portion of page names in the Mediawiki. We'll do that with Apache's path rewriting engine.

  • Enable the rewriting module
 root@server:~# a2enmod rewrite
  • Modify the /etc/apache2/sites-available/site.domain.tld file, in the place specified above.
RewriteEngine on
RewriteCond %{REQUEST_URI} !^/(stylesheets|images|skins)/
RewriteCond %{REQUEST_URI} !^/(redirect|texvc|index|load|api).php
RewriteCond %{REQUEST_URI} !^(load|api).php
RewriteCond %{REQUEST_URI} !^/error/(40(1|3|4)|500).html
RewriteCond %{REQUEST_URI} !^/favicon.ico
RewriteCond %{REQUEST_URI} !^/robots.txt
RewriteCond %{REQUEST_URI} !^/sitemap/
RewriteCond %{REQUEST_URI} !^/.well-known/
RewriteCond %{REQUEST_URI} !^/mediawiki/

# Rewrite http://wiki.domain.tld/article properly, this is the main rule
RewriteRule ^/(.*)$ %{DOCUMENT_ROOT}/mediawiki/index.php/?title=$1 [L,QSA]
  • Reload Apache's confiuration
root@server:~# /etc/init.d/apache2 reload
  • Modify LocalSettings.php, just under the $wgScriptPath, by adding the settings below:
$wgScriptPath = "/mediawiki";

## Added for short names
$wgScriptExtension = ".php";
$wgArticlePath      = "/$1";
$wgUsePathInfo      = false;

You should now be able to go to http://site.domain.tld/Main_Page and see the main page of the wiki.

Tweaking Mediawiki

There are still a few things that need doing in Mediawiki to smooth off some rough edges.

Restricting editing

To prevent vandalism of the wiki, we want to restrict account creation to sysops, and prevent any anonymous editing of the wiki. Add these lines to the end of the LocalSettings.php file:

# Prevent vandalism of the wiki

# Prevent new user registrations except by sysops
$wgGroupPermissions['*']['createaccount'] = false; 

# Disable anonymous editing
$wgGroupPermissions['*']['edit'] = false;

Restricting viewing

If you want to restrict access to the wiki, add the following to LocalSettings.php to ensure that people can only access the wiki's main page.

# Disable anonymous browsing
$wgWhitelistRead = array( "Main Page", "Special:Userlogin", "-", 
    "MyWiki:Privacy policy", "MyWiki:About", "MyWiki:General disclaimer");
$wgGroupPermissions['*']['read'] = false;

Robots file and favicon

Simply put these files in the document root, /var/www/site.domain.tld.

Enabling uploads

In LocalSettings.php, modify the settings around $wgEnableUploads to be:

## To enable image uploads, make sure the 'images' directory
## is writable, then set this to true:
$wgEnableUploads = true;
$wgUseImageMagick = true;
$wgImageMagickConvertCommand = "/usr/bin/convert";
$wgUploadDirectory = "/var/www/domain.tld/mediawiki/images";
$wgUploadPath = "{$wgScriptPath}/images";

## The file path for temporary files. Was /tmp
$wgTmpDirectory = "/var/www/domain.tld/mediawiki/images/temp";
$wgGenerateThumbnailOnParse = true;

Yes, again, there need to be explicit absolute paths on the server's filesystem.

Extending uploadable files

Out of the box, Mediawiki only allows the uploads of image files. It allow more file types to be uploaded, include this line in LocalSettings.php

## Allow more file types to be uploaded
$wgFileExtensions = array_merge( $wgFileExtensions, array( 'pdf', 'doc', 'xls', 'ppt', 'docx', 'xlsx', 'pptx', 'odt', 'odc', 'odp', 'odg', 'odi', 'gz', 'zip' ) );

You'll need to do this for each Mediawiki instance. There are also some MIME types that Mediawiki doesn't know about. Edit these lines in /usr/share/mediawiki/includes/mime.types:

application/x-gzip gz tgz # Added tgz

(Following the instructions at MediaWiki.org on $wgFileExtensions and MIME type validation.)

  • Create your logo as a 135x135 px PNG file.
  • Upload it to the wiki.
  • Open the file in Mediawiki to find the path where Mediawiki has stored it: it will be something like http://site.domain.tld/mediawiki/images/7/77/Logo.png : note the path name.
  • Modify the LocalSettings.php file as shown.
$wgLogo = "{$wgScriptPath}/images/7/77/Logo.png";

Enabling equation display

Download and install the Math extension, following the instructions given. They basically boil down to downloading the extension file, uncompressing it in the /var/lib/mediawiki/extensions directory, then loading it.

root@server:~# cd /var/lib/mediawiki/extensions/Math/
# install the dependencies listed in the README file e.g.:
root@server:/var/lib/mediawiki/extensions/Math# aptitude install build-essential ocaml dvipng texlive-fonts-recommended texlive-lang-greek texlive-latex-recommended
root@server:/var/lib/mediawiki/extensions/Math# make
  • Add these lines to the end of LocalSettings.php
wfLoadExtension( 'Math' );
// Enable the various rendering modes
# $wgMathValidModes[] = 'mathml';
$wgMathValidModes[] = 'latex';
// Set the default rendering option
# $wgDefaultUserOptions['math'] = 'mathml';
# $wgMathFullRestbaseURL= 'https://api.formulasearchengine.com/';
wgDefaultUserOptions['math'] = 'latex';
  • While testing, you might want to add this to LocalSettings.php as well:
$wgShowExceptionDetails = true;
  • In the shell, run the update script, for each wiki in the farm:
root@server:~# cd /var/lib/mediawiki/maintenance 
root@server:/var/lib/mediawiki/maintenance# sudo -u www-data php update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php

Generate sitemaps

Sitemaps help search engines, particuarly Google, index sites. Mediawiki comes with a standard script for generating sitemaps.

  • Create a directory for the sitemap:
root@server:~# mkdir /var/www/domain.tld/sitemap
root@server:~# chown www-data:www-data /var/www/domain.tld/sitemap
  • If you're rewriting paths, don't rewrite the path to the sitemap. Edit /etc/apache2/sites-enabled/domain.tld.conf to include the line
RewriteCond %{REQUEST_URI} !^/sitemap/
then reload the server.
root@server:~# systemctl reload apache2.service
  • Create a cron job as </etc/cron.d/mediawiki-sitemaps to regenerate the sitemap daily. (This needs to run the script as www-data user, as the script rewrites the Mediawiki cache. If it's done as root, the cache files become unwritable by the web server and Mediawiki stops with an error.)
07 01 * * * www-data php /var/www/domain.tld/mediawiki/maintenance/generateSitemap.php \
   --fspath /var/www/domain.tld/sitemap \
   --server https://www.domain.tld \
   --urlpath https://www.domain.tld/sitemap \
   --conf /var/www/domain.tld/mediawiki/LocalSettings.php
  • Submit the sitemap to the search engine.

Repeat these steps for every wiki in the farm.

Installing 'Help' pages

Basic Mediawiki installations don't come with any help pages. To copy the help pages from an existing wiki, go to the 'Special Pages':'Export' page on the existing wiki, and export the following set of pages:

  • Help:Contents
  • Help:Navigation
  • Help:Searching
  • Help:Tracking_changes
  • Help:Editing_pages
  • Help:Starting_a_new_page
  • Help:Formatting
  • Help:Links
  • Help:Categories
  • Help:Images
  • Help:Templates
  • Help:Tables
  • Help:Variables
  • Help:Managing_files
  • Help:Preferences
  • Help:Skins
  • Help:Namespaces
  • Help:Interwiki_linking (don't include)
  • Help:Special pages
  • Template:PD_Help_Page
  • Template:Admin_tip
  • Template:Prettytable
  • Template:Hl2
  • Template:Hl3
  • Template:Thankyou
  • Image:Example.jpg
  • Image:Geographylogo.png
  • Image:Tools.png
  • Image:M-en-sidebar.png
  • Image:M-en-pagetabs.png
  • Image:M-en-userlinks.png
  • Image:M-en-recentchanges.png
  • Template:Click
  • Template:Languages
  • Help:Range_blocks
  • Help:Managing_user_rights
  • Help:Copying
  • Category:Help
  • Category:Category

You'll also need to download the images themselves. Then, import the pages into the new Mediawiki and upload the pages. You'll probably want to modify the PD_Help_Page template.

Installing a mobile skin

The Mediawiki extension MobileFrontend gives a different presentation for mobile devices. To install, follow the instructions on the MobileFrontend page, placing the extension's .tar.gz file in /var/lib/mediawiki/extensions and expanding it there.

Then modify LocalSettings.php to include, at the end

# Enable mobile front end

wfLoadExtension( 'MobileFrontend' );
$wgMFAutodetectMobileView = true;
# wfLoadSkin( 'MinervaNeue' );
$wgMFDefaultSkinClass = 'SkinMinerva'; // use Minerva skin

Note that in Mediawiki 1.27 LTS, there is no separate MinervaNeue skin outside the MobileFrontend extension, so no need to install that. (The separate skin comes in from version 1.30 onwards).

Restoring, and upgrading from previous versions

  • Take a backup of the wiki database
 root@server:~# mysqldump --user='backup' --opt --databases $db | bzip2 > "${BACKUPFILENAME}.${db}-dump.sql.bz2"
  • If necessary, create a new, empty database and the user to run it:
root@server:~# mysql -u root -p
mysql> create database wikidb;
mysql> create user 'wikiuser'@'localhost' identified by 'wikiuserpassword';
mysql> grant all on wikidb.* to 'wikiuser'@'localhost';
mysql> quit;
  • Load the database
root@server:~# bzcat DB_DUMP_FILE | mysql -u root -p
  • Copy all the files from the backup into /var/www/site.domain.tld/mediawiki/
root@server:~# cp -pr /path/to/backup/site.domain.tld /var/www/
  • Delete /var/www/site.domain.tld/mediawiki/Test.php
  • Rename the LocalSettings.php file to LocalSettings.php.old
  • Move the uploads direcory to its new name images
root@server:~# mv /var/www/site.domain.tld/mediawiki/uploads /var/www/site.domain.tld/mediawiki/images
  • Make a few changes in LocalSettings.php
ini_set( 'memory_limit'. '128M' );
$wgDBadminuser           = "root";
$wgDBadminpassword       = "RootUserPassword";
wgUploadPath       = "$wgScriptPath/images";
$wgUploadDirectory  = "$IP/images";
  • Update the database
root@server:~# php /usr/share/mediawiki/maintenance/update.php --conf /var/www/site.domain.tld/mediawiki/LocalSettings.php
  • Remove the $wgDBadminuser and $wgDBadminpassword lines from LocalSettings.php
  • Add the config files to Apaches
root@server:~# cp /path/to/backup/etc/apache2/sites-available/site.domain.tld /etc/apache2/sites-available/
root@server:~# a2ensite site.domain.tld
root@server:~# service apache2 reload

See also

Here are a few pages that are useful guides or provide background and context.