Deploying a source install

Once you’ve installed CKAN from source by following the instructions in Installing CKAN from source, you can follow these instructions to deploy your CKAN site using a production web server (Apache), so that it’s available to the Internet.

Note

If you installed CKAN from package you don’t need to follow this section, your site is already deployed using Apache and modwsgi as described below.

Because CKAN uses WSGI, a standard interface between web servers and Python web applications, CKAN can be used with a number of different web server and deployment configurations including:

  • Apache with the modwsgi Apache module proxied with Nginx for caching
  • Apache with the modwsgi Apache module
  • Apache with paster and reverse proxy
  • Nginx with paster and reverse proxy
  • Nginx with uwsgi

This guide explains how to deploy CKAN using Apache and modwsgi and proxied with Nginx on an Ubuntu server. These instructions have been tested on Ubuntu 12.04.

If run into any problems following these instructions, see Troubleshooting below.

1. Create a production.ini File

Create your site’s production.ini file, by copying the development.ini file you created in Installing CKAN from source earlier:

cp /etc/ckan/default/development.ini /etc/ckan/default/production.ini

2. Install Apache, modwsgi, modrpaf

Install Apache (a web server), modwsgi (an Apache module that adds WSGI support to Apache), and modrpaf (an Apache module that sets the right IP address when there is a proxy forwarding to Apache):

sudo apt-get install apache2 libapache2-mod-wsgi libapache2-mod-rpaf

3. Install Nginx

Install Nginx (a web server) which will proxy the content from Apache and add a layer of caching:

sudo apt-get install nginx

4. Install an email server

If one isn’t installed already, install an email server to enable CKAN’s email features (such as sending traceback emails to sysadmins when crashes occur, or sending new activity email notifications to users). For example, to install the Postfix email server, do:

sudo apt-get install postfix

When asked to choose a Postfix configuration, choose Internet Site and press return.

5. Create the WSGI script file

Create your site’s WSGI script file /etc/ckan/default/apache.wsgi with the following contents:

import os
activate_this = os.path.join('/usr/lib/ckan/default/bin/activate_this.py')
execfile(activate_this, dict(__file__=activate_this))

from paste.deploy import loadapp
config_filepath = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'production.ini')
from paste.script.util.logging_config import fileConfig
fileConfig(config_filepath)
application = loadapp('config:%s' % config_filepath)

The modwsgi Apache module will redirect requests to your web server to this WSGI script file. The script file then handles those requests by directing them on to your CKAN instance (after first configuring the Python environment for CKAN to run in).

6. Create the Apache config file

Create your site’s Apache config file at /etc/apache2/sites-available/ckan_default.conf, with the following contents:

<VirtualHost 127.0.0.1:8080>
    ServerName default.ckanhosted.com
    ServerAlias www.default.ckanhosted.com
    WSGIScriptAlias / /etc/ckan/default/apache.wsgi

    # Pass authorization info on (needed for rest api).
    WSGIPassAuthorization On

    # Deploy as a daemon (avoids conflicts between CKAN instances).
    WSGIDaemonProcess ckan_default display-name=ckan_default processes=2 threads=15

    WSGIProcessGroup ckan_default

    ErrorLog /var/log/apache2/ckan_default.error.log
    CustomLog /var/log/apache2/ckan_default.custom.log combined

    <IfModule mod_rpaf.c>
        RPAFenable On
        RPAFsethostname On
        RPAFproxy_ips 127.0.0.1
    </IfModule>

    <Directory />
        Require all granted
    </Directory>

</VirtualHost>

Replace default.ckanhosted.com and www.default.ckanhosted.com with the domain name for your site.

Note

If you are running Apache 2.2 or lower (eg on Ubuntu 12.04), remove this directive, as it is not supported:

<Directory />
    Require all granted
</Directory>

This tells the Apache modwsgi module to redirect any requests to the web server to the WSGI script that you created above. Your WSGI script in turn directs the requests to your CKAN instance.

7. Modify the Apache ports.conf file

Open /etc/apache2/ports.conf. We need to replace the default port 80 with the 8080 one.

  • On Apache 2.4 (eg Ubuntu 14.04 or RHEL 7):

    Replace this line:

    Listen 80

    With this one:

    Listen 8080

  • On Apache 2.2 (eg Ubuntu 12.04 or RHEL 6):

    Replace these lines:

    NameVirtualHost *:80
Listen 80

    With these ones:

    NameVirtualHost *:8080
Listen 8080

8. Create the Nginx config file

Create your site’s Nginx config file at /etc/nginx/sites-available/ckan, with the following contents:

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m max_size=250m;
proxy_temp_path /tmp/nginx_proxy 1 2;

server {
    client_max_body_size 100M;
    location / {
        proxy_pass http://127.0.0.1:8080/;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_cache cache;
        proxy_cache_bypass $cookie_auth_tkt;
        proxy_no_cache $cookie_auth_tkt;
        proxy_cache_valid 30m;
        proxy_cache_key $host$scheme$proxy_host$request_uri;
        # In emergency comment out line to force caching
        # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
    }

}

9. Enable your CKAN site

To prevent conflicts, disable your default nginx and apache sites. Finally, enable your CKAN site in Apache:

sudo a2ensite ckan_default
sudo a2dissite 000-default
sudo rm -vi /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/ckan /etc/nginx/sites-enabled/ckan_default
sudo service apache2 reload
sudo service nginx reload

You should now be able to visit your server in a web browser and see your new CKAN instance.

10. Setup a worker for background jobs

CKAN uses asynchronous Background jobs for long tasks. These jobs are executed by a separate process which is called a worker.

To run the worker in a robust way, install and configure Supervisor.

Troubleshooting

Default Apache welcome page

If you see a default Apache welcome page where your CKAN front page should be, it may be because the default Apache config file is overriding your CKAN config file (both use port 80), so disable it and restart Apache:

sudo a2dissite default
sudo service apache2 reload

403 Forbidden and 500 Internal Server Error

If you see a 403 Forbidden or 500 Internal Server Error page where your CKAN front page should be, you may have a problem with your unix file permissions. The Apache web server needs to have permission to access your WSGI script file and all of its parent directories. The permissions of the file should look like -rw-r--r-- and the permissions of each of its parent directories should look like drwxr-xr-x.

IOError: sys.stdout access restricted by mod_wsgi

If you’re getting 500 Internal Server Error pages and you see IOError: sys.stdout access restricted by mod_wsgi in your log files, it means that something in your WSGI application (e.g. your WSGI script file, your CKAN instance, or one of your CKAN extensions) is trying to print to stdout, for example by using standard Python print statements. WSGI applications are not allowed to write to stdout. Possible solutions include:

  1. Remove the offending print statements. One option is to replace print statements with statements like print >> sys.stderr, "..."

  2. Redirect all print statements to stderr:

    import sys
sys.stdout = sys.stderr

  3. Allow your application to print to stdout by putting WSGIRestrictStdout Off in your Apache config file (not recommended).

Also see https://code.google.com/p/modwsgi/wiki/ApplicationIssues

Log files

In general, if it’s not working look in the log files in /var/log/apache2 for error messages. ckan_default.error.log should be particularly interesting.