umap/docs/ubuntu.md

# Tutorial

This tutorial will cover an installation from scratch of a uMap instance in an Ubuntu server.

You need sudo grants on this server, and it must be connected to Internet.

## Install system dependencies

    sudo apt install python3 python3-dev python3-venv wget nginx uwsgi uwsgi-plugin-python3 postgresql gcc postgis libpq-dev

*Note: nginx and uwsgi are not required for local development environment*

## Create deployment directories:

   sudo mkdir -p /etc/umap

*You can change this path, but then remember to adapt the other steps accordingly.*


## Create a Unix user

    sudo useradd -N umap -m -d /srv/umap/

*Here we use the name `umap`, but this name is up to you. Remember to change it
on the various commands and configuration files if you go with your own.*


## Give umap user access to the config folder

    sudo chown umap:users /etc/umap
    sudo chown umap:users /srv/umap


## Create a postgresql user

    sudo -u postgres createuser umap


## Create a postgresql database

    sudo -u postgres createdb umap -O umap


## Activate PostGIS extension

    sudo -u postgres psql umap -c "CREATE EXTENSION postgis"


## Login as umap Unix user

    sudo -u umap -i

From now on, unless we say differently, the commands are run as `umap` user.


## Create a virtualenv and activate it

    virtualenv /srv/umap/venv --python=/usr/bin/python3.6
    source /srv/umap/venv/bin/activate

*Note: this activation is not persistent, so if you open a new terminal window,
you will need to run again this last line.*


## Install umap

    pip install umap-project


## Create a local configuration file

    wget https://raw.githubusercontent.com/umap-project/umap/master/umap/settings/local.py.sample -O /etc/umap/umap.conf

## Customize umap.conf

    nano /etc/umap/umap.conf

Change the following properties:

```
STATIC_ROOT = '/srv/umap/var/static'
MEDIA_ROOT = '/srv/umap/var/data'
```

## Create the tables

    umap migrate

## Collect the statics

    umap collectstatic

## Create a superuser

    umap createsuperuser

## Start the demo server

    umap runserver 0.0.0.0:8000

You can now go to [http://localhost:8000/](http://localhost:8000/) and try to create a map for testing.

When you're done with testing, quit the demo server (type Ctrl+C).


## Configure the HTTP API

Now let's configure a proper HTTP server.

### uWSGI

Create a file named `/srv/umap/uwsgi_params`, with this content
(without making any change on it):

```
uwsgi_param  QUERY_STRING       $query_string;
uwsgi_param  REQUEST_METHOD     $request_method;
uwsgi_param  CONTENT_TYPE       $content_type;
uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;
uwsgi_param  PATH_INFO          $document_uri;
uwsgi_param  DOCUMENT_ROOT      $document_root;
uwsgi_param  SERVER_PROTOCOL    $server_protocol;
uwsgi_param  REQUEST_SCHEME     $scheme;
uwsgi_param  HTTPS              $https if_not_empty;

uwsgi_param  REMOTE_ADDR        $remote_addr;
uwsgi_param  REMOTE_PORT        $remote_port;
uwsgi_param  SERVER_PORT        $server_port;
uwsgi_param  SERVER_NAME        $server_name;
```

Then create a configuration file for uWSGI:

    nano /srv/umap/uwsgi.ini

And paste this content. Double check paths and user name in case you
have customized some of them during this tutorial. If you followed all the bits of the
tutorial without making any change, you can use it as is:

```
[uwsgi]
uid = umap
gid = users
# Python related settings
# the base directory (full path)
chdir           = /srv/umap/
# umap's wsgi module
module          = umap.wsgi
# the virtualenv (full path)
home            = /srv/umap/venv

# process-related settings
# master
master          = true
# maximum number of worker processes
processes       = 4
# the socket (use the full path to be safe
socket          = /srv/umap/uwsgi.sock
# ... with appropriate permissions - may be needed
chmod-socket    = 666
stats           = /srv/umap/stats.sock
# clear environment on exit
vacuum          = true
plugins         = python3

```

### Nginx

Create a new file:

    nano /srv/umap/nginx.conf

with this content:

```
# the upstream component nginx needs to connect to
upstream umap {
    server unix:///srv/umap/uwsgi.sock;
}

# configuration of the server
server {
    # the port your site will be served on
    listen      80;
    listen   [::]:80;
    listen      443 ssl;
    listen   [::]:443 ssl;
    # the domain name it will serve for
    server_name your-domain.org;
    charset     utf-8;

    # max upload size
    client_max_body_size 5M;   # adjust to taste

    # Finally, send all non-media requests to the Django server.
    location / {
        uwsgi_pass  umap;
        include     /srv/umap/uwsgi_params;
    }
}
```

Remember to adapt the domain name.

### Activate and restart the services

Now quit the `umap` session, simply by typing Ctrl+D.

You should now be logged in as your normal user, which is sudoer.

- Activate the Nginx configuration file:

        sudo ln -s /srv/umap/nginx.conf /etc/nginx/sites-enabled/umap

- Activate the uWSGI configuration file:

        sudo ln -s /srv/umap/uwsgi.ini /etc/uwsgi/apps-enabled/umap.ini

- Restart both services:

        sudo systemctl restart uwsgi nginx


Now you should access your server through your url and create maps:

    http://yourdomain.org/


Congratulations!

- - -

## Troubleshooting

- Nginx logs are in /var/log/nginx/:

        sudo tail -f /var/log/nginx/error.log
        sudo tail -f /var/log/nginx/access.log

- uWSGI logs are in /var/log/uwsgi:

        sudo tail -f /var/log/uwsgi/umap.log


## Before going live

### Add a real SECRET_KEY

In your local.py file, add a real secret and unique `SECRET_KEY`, and do
not share it.

### Remove DEMO flag

In your local.py:

    UMAP_DEMO_SITE = False
    DEBUG = False

### Configure Nginx to serve statics and uploaded files:

In your nginx config:

    location /static {
        autoindex off;
        access_log off;
        log_not_found off;
        sendfile on;
        gzip on;
        gzip_vary on;
        alias /path/to/umap/var/static/;   
    }

    location /uploads {
        autoindex off;
        sendfile on;
        gzip on;
        gzip_vary on;
        alias /path/to/umap/var/data/;
        # Exclude direct acces to geojson, as permissions must be
        # checked py django.
        location /uploads/datalayer/ { return 404; }
    }

### Configure social auth

Now you can login with your superuser, but you may allow users to user social
authentication.

### Configure default map center

In your local.py change those settings:

    LEAFLET_LONGITUDE = 2
    LEAFLET_LATITUDE = 51
    LEAFLET_ZOOM = 6

### Activate statics compression

In your local.py, set `COMPRESS_ENABLED = True`, and then run the following command

    umap compress

Optionally add `COMPRESS_STORAGE = "compressor.storage.GzipCompressorFileStorage"`
and add `gzip_static on` directive to Nginx `/static` location, so Nginx will
serve pregenerated files instead of compressing them on the fly.

### Configure the site URL and short URL

In your local.py:

    SITE_URL = "http://localhost:8019"
    SHORT_SITE_URL = "http://s.hort"

Also adapt `ALLOWED_HOSTS` accordingly.

### Configure X-Accel-Redirect

In order to let Nginx serve the layer geojsons but uMap still check the permissions,
you can add this settings:

    UMAP_XSENDFILE_HEADER = 'X-Accel-Redirect'

And then add this new location in your nginx config (before the `/` location):

    location /internal/ {
        internal;
        gzip_vary on;
        gzip_static on;
        alias /path/to/umap/var/data/;
    }


### Configure ajax proxy cache

uMap allows to use remote URL as data sources, but those URLs are not always
CORS open, so this is why there is this "ajax-proxy" feature, where the URL is
passed to the backend.

Additionally, there is a caching feature, which duration is configurable through
frontend settings. Valid values are: disabled, 5 min, 1 hour, 1 day.

This configuration provides a mix option, where python deals with validating the
URL and parsing the TTL parameter, and then it passes the hand to nginx which
will serve the remote content.

So, roughly:

- the client calls `/ajax-proxy/?url=xxx&ttl=300`
- python will validate the URL (not internal calls…)
- if `UMAP_XSENDFILE_HEADER` is set, then the python returns an empty response
  with the path `/proxy/http://url` plus it will set the cache TTL through the
  header `X-Accel-Expires`
- this `/proxy/` location is then handled by nginx


In Nginx:

- add the proxy cache

        proxy_cache_path /tmp/nginx_ajax_proxy_cache levels=1:2 keys_zone=ajax_proxy:10m inactive=60m;
        proxy_cache_key "$uri$is_args$args";

- add those locations (before the `/` location):

        location ~ ^/proxy/(.*) {
            internal;
            add_header X-Proxy-Cache $upstream_cache_status always;
            proxy_cache ajax_proxy;
            proxy_cache_valid 1m;  # Default. Umap will override using X-Accel-Expires
            set $target_url $1;
            # URL is encoded, so we need a few hack to clean it back.
            if ( $target_url ~ (.+)%3A%2F%2F(.+) ){ # fix :// between scheme and destination
              set $target_url $1://$2;
            }
            if ( $target_url ~ (.+?)%3A(.*) ){ # fix : between destination and port
              set $target_url $1:$2;
            }
            if ( $target_url ~ (.+?)%2F(.*) ){ # fix / after port, the rest will be decoded by proxy_pass
              set $target_url $1/$2;
            }
            resolver 8.8.8.8;
            add_header X-Proxy-Target $target_url; # For debugging
            proxy_read_timeout 10s;
            proxy_connect_timeout 5s;
            proxy_pass $target_url;
            proxy_intercept_errors on;
            error_page 301 302 307 = @handle_proxy_redirect;
        }
        location @handle_proxy_redirect {
            resolver 8.8.8.8;
            set $saved_redirect_location '$upstream_http_location';
            proxy_pass $saved_redirect_location;
        }




## Add more tilelayers, pictograms…

Go to the admin: [http://localhost:8020/admin/](http://localhost:8020/admin/),
and create the objects you want.