jeff/umap
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[docs] Update the documentation

Browse source
This commit is contained in:
Alexis Métaireau 2023-11-28 15:54:11 +01:00
parent 34ee8e81de
commit b3d6d6da08
11 changed files with 311 additions and 339 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

73
docs/architecture/frontend.md Normal file
View file

@ -0,0 +1,73 @@
# The frontend
uMap is using Leaflet on the frontend. It's actually extending it: each uMap object extends the ones from Leaflet.
```mermaid
erDiagram
Map ||--o{ DataLayer : datalayers
Map{
JSON permissions
JSON metadata
}
DataLayer ||--o{Feature : data
DataLayer {
JSON metadata
JSON permissions
}
```
Here are the important concepts and files:
- `umap.js` contains `Map`, the map object. It's the main entry point, which references all the related properties. It contains metadata, permissions, and data layers.
- `umap.layer.js` contains `DataLayer`, which contains metadata, permissions, and data.
- `umap.permissions.js` handles the permissions of the map. There is a different file handling the permissions of the datalayer:
- `umap.datalayer.permissions.js`.
## Map (`L.U.Map`)
`L.U.Map` is the class that's called by the server templates (in `map_init.html` and `map_fragment.html` used when we display lists of maps, like the homepage).
It contains references to datalayers, and to the controls (the buttons that appears on the map)
To be able to change the way the client behaves, querystring parameters can be used to overload the settings stored in the database. This is useful for instance when using iframes to display the map on a different website.
uMap has an edit mode. If you don't have the rights you cannot save nor edit, you can't edit the permissions as well.
A map contains references to:
- controls
- datalayers
## DataLayers (`L.U.Datalayer`)
The datalayers contains data, and a layer (a way to represent them).
Each data layer contains a "layer", to know what type of layer it is. It's one of:
- Choropleth (`L.U.Layer.Choropleth`)
- Cluster (`L.U.Layer.Cluster`)
- Heat (`L.U.Layer.Heat`)
When the data layers are initialized, they can have two states:
- `loaded`: the object is loaded in memory. At this stage we have access to all the datalayer's metada (name, type, id)
- `dataLoaded` : the data is available to be used, so we can for instance compute the center of the map (when it's dynamic).
- `backupOptions` is here to make it possible to cancel what have been done (using undo). It stores the old settings for the data-layer.
## Storage
To mark what needs to be synced with the server, uMap currently mark objects as "dirty". Something marked as "dirty" has changed on the client, but is not yet saved on the server.
Each map, datalayer and permission objects can be marked as "dirty". When a change is made on an object, it will mark its parent as "dirty" as well, so it can be updated accordingly.
### Saving data to the server with `umap.save()`
Here is what's being done when you call `map.save()`:
1. `map.saveSelf()`, posting `name`, `center` and `settings` to the server, and then
2. calls `permission.save()`, which will post the permissions to the server, and then call back
3. `map.continueSaving()`, which will take the first dirtyLayer and call
4. `datalayer.save()` on it. It does the following:
1. Post the data (`name`, `displayOnLoad`, `rank`, `settings`, and `geojson`)
2. Calls `permission.save()`, posting `edit_status` to the server, and then calling `map.continue_saving()` and remove the datalayer from `dirtyDatalayers`.
5. When the `dirtyDatalayers` list is empty, we are done.

91
docs/contributing.md
View file

@ -1,92 +1,31 @@
# Contributing
So you want to contribute to uMap? Great news 🙌
We've put together this document so you have a brief overview of how things work.
You can help on different areas: translation, bug triage, documentation and development.
## Translating
Translation is managed through [Transifex](https://www.transifex.com/openstreetmap/umap/).
uMap is translated to more than 50 languages! The translation is managed through [Transifex](https://www.transifex.com/openstreetmap/umap/). You will need an account to get started, and then you'll be able to translate easily.
## Bug Triaging
## Bug Triage
You are very welcome to help us triaging [uMap issues](https://github.com/umap-project/umap/issues).
You are very welcome to help us triage [uMap issues](https://github.com/umap-project/umap/issues). Don't hesitate to help other users by answering questions, give your point of view in discussions and just report bugs!
* Help other users by answering questions
* Give your point of view in discussions
* And so on...
## Reporting a bug
## Development on Ubuntu
If you've encountered a bug, don't hesitate to tell us about it. The best way to do this is by [opening a ticket on the bug tracker](https://github.com/umap-project/umap/issues/new/choose). But please, first, [have a look around](https://github.com/umap-project/umap/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc) to see if other users already reported something 😅
### Environment setup
## Hacking on the code
Choose one of the following two config:
Following the [installation instructions](install.md) should get you started to hack on the code.
#### Config global to your desktop
## Merging rules
Follow the procedure [Ubuntu from scratch](ubuntu.md)
Pull requests
But instead using folders /etc/umap, you can create a ~/.umap folder.
This folder will contain the umap.conf file.
And for folder /srv/umap, you can create a ~/umap folder (We will remove this folder later)
You will have to set an env var, we will set it in your .bashrc:
nano ~/.bashrc
Add the following at the end of file:
```
# uMap
export UMAP_SETTINGS=~/.umap/umap.conf
```
Then refresh your terminal
source ~/.bashrc
Run your local uMap and check that it is working properly.
#### Config inside your local git repo
Follow the procedure [Ubuntu from scratch](ubuntu.md)
You can use the local.py.sample in the git repo and copy it to your local git repo to umap/settings/local.py
See [Installation](install.md)
### Hacking on the code
Create a workspace folder `~/wk` and go into it.
"git clone" the main repository and go in the `umap/` folder.
Once you are in the `umap/` folder, create a Python virtual environment:
python3 -m venv venv
source venv/bin/activate
Now, the `umap` command will be available.
*Note: if you close your terminal, you will need to re-run that command from `~/wk/umap`:*
source venv/bin/activate
To test your code, you will add to install umap from your git folder. Go to `~/wk/umap` and run:
make install
This command will check dependencies and install uMap from sources inside folder.
When installing from the git repository, do not forget to run `make installjs` and `make vendors`, before running `umap collectstatic` (as mentioned in [Ubuntu from scratch](ubuntu.md)).
Create a PostgreSQL database and apply migrations to setup your database:
createdb umap
umap migrate
You should now be able to start your local uMap instance:
umap runserver 0.0.0.0:8000
### Update translations
## Update translations
Install needed tools:

1
docs/custom.md
View file

@ -1,6 +1,5 @@
# Customize your uMap installation
When running your own uMap, you may want to changed its appearance, for example
you want your own logo on the home page, or you want to apply some design, or
you want to add some tracking (but anonymous!) script…

51
docs/docker.md → docs/deploy/docker.md
View file

@ -1,11 +1,14 @@
# Docker
There is now an official [uMap](https://hub.docker.com/r/umap/umap) image.
## Docker Hub
To run it with docker compose, create a `docker-compose.yml` like this:
An official [uMap docker image](https://hub.docker.com/r/umap/umap) is available on the docker hub.
_Sample docker-compose.yml_
```yaml
## Docker compose
If you prefer to run it with docker compose, here is the configuration file:
```yaml title="docker-compose.yml"
version: '3'
services:
db:
@ -47,16 +50,17 @@ You can use this example below and it will run, but you may want to look at the
Make sure the settings in the docker-compose don't conflict with the sample config and vice-versa. In particular, remove the DATABASES section from the config file if using the docker-compose file, or it will override the DATABASE_URL setting and things won't work.
_Sample umap.conf_
```python
```python title="umap.conf"
"""
Example settings for docker quickstart:
Lots of stuff has been removed for simplicity
Example settings for docker quickstart: lots of stuff has been removed for simplicity.
See https://umap-project.readthedocs.io/en/master/settings/
You can get the whole list of settings at:
Things YOU need to change before launching:
- modify SECRET_KEY
https://umap-project.readthedocs.io/en/master/settings/
Here are the settings YOU HAVE TO CHANGE before launching:
- SECRET_KEY
"""
from umap.settings.base import * # pylint: disable=W0614,W0401
@ -88,32 +92,15 @@ Some basic settings are available through env vars (see https://github.com/umap-
but if you need more custom ones (like custom OAuth configuration), the easiest
way is to put them in a [settings file](settings.md) and mount it to `/etc/umap/umap.conf`.
---
### Getting started with docker compose
#### Quick sidebar about docker and docker-compose
A multi-platform tutorial on [installing docker](https://docs.docker.com/get-docker/) and docker-compose is outside the scope of this howto, but if you are on a recent Ubuntu, this is all you need to install both:
With docker installed on your machine, start the server with
```bash
sudo groupadd docker
sudo snap install docker
sudo usermod -aG docker $USER
# EXIT and log back in to pickup the docker group membership, then test with
docker run hello-world
```
Current versions of docker compose are integrated into the docker command and launch with `docker compose` (two words) -- older versions use a separate binary named `docker-compose`. Either one will work for this.
---
### Initial launch
Start the server with
```bash
docker compose up
```
... and let it run some initial setup until the output quiesces with a message about spawning uWSGI workers. Because there is a race between the time the app tries to connect to the DB and when the DB is actually ready, you might see a few exceptions/errors about 'psycopg' being unable to connect. This should sort itself out as the app retries.
### Create superuser
... and let it run some initial setup until the output quiesces with a message about spawning uWSGI workers. Because there is a race between the time the app tries to connect to the DB and when the DB is actually ready, you might see a few exceptions/errors about 'psycopg' being unable to connect. This should sort itself out as the app retries.
Now you need to create your site superuser. Stop the server (Ctrl-C) and then type:
```bash
@ -122,8 +109,6 @@ docker-compose run app /venv/bin/umap createsuperuser
Once that's done, you can relaunch your server with `docker compose up`
### Try It!
You should now be able to browse to your uMap instance from a browser on your local system, by pointing your browser to `https://localhost:8001/` (equivalent to `${SITE_URL}` in the docker-compose file, above).
### Administration

8
docs/ubuntu.md → docs/deploy/ubuntu.md
View file

@ -6,14 +6,20 @@ You need sudo grants on this server, and it must be connected to Internet.
## Install system dependencies
``` bash
sudo apt update
sudo apt install python3 python3-dev python3-venv virtualenv wget nginx uwsgi uwsgi-plugin-python3 postgresql gcc postgis libpq-dev
```
*Note: nginx and uwsgi are not required for local development environment*
!! note
Nginx and uwsgi are not required for local development environment.
## Create deployment directories:
```bash
sudo mkdir -p /etc/umap
```
*You can change this path, but then remember to adapt the other steps accordingly.*

17
docs/index.md
View file

@ -1,8 +1,17 @@
# uMap developer documentation
*This documentation is intended for people aiming to install and configure uMap.
If you are looking for user documentation, have a look at [the OSM wiki page](http://wiki.openstreetmap.org/wiki/UMap#Tutorials).*
!!! info "Developer docs"
uMap is a Django project, so in case of doubt always refer to the [Django documentation](https://docs.djangoproject.com).
This documentation is intended for people aiming to install and configure uMap.
Now let's start by [installing uMap](install.md)!
If you are looking for user documentation, have a look at [the OSM wiki page](http://wiki.openstreetmap.org/wiki/UMap#Tutorials).
uMap lets you create maps with OpenStreetMap layers in a minute, and embed them in your site.
If you want to get started, follow the [installation](install.md) guide.
You might want to:
- See [the developer docs](developer.md)
- Read [the contributing guidelines](contributing.md)
- Read through the [deployment tips](deploy/tips.md)

129
docs/install.md
View file

@ -1,43 +1,104 @@
# Installation
*Note: for Ubuntu, follow procedure [Ubuntu from scratch](ubuntu.md)*
!!! info "Specific guides"
*Note: for Windows, follow procedure [Installing on Windows](install_windows.md)*
This page covers how to get started with uMap. If you plan on deploying the service on a server for other people to consume, you can follow different tutorials:
Create a geo aware database. See [Geodjango doc](https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/) for backend installation.
- [Deploying uMap on Ubuntu](deploy/ubuntu.md) (from scratch)
- [Deploying uMap using Docker](deploy/docker.md)
- Recent Windows distribution provide [WSL 2](https://learn.microsoft.com/en-us/windows/wsl/install), which makes it possible to run a Linux distribution of your choice.
Create a virtual environment
## System dependencies
uMap is built with the [Python](https://python.org) language, and the [Django](https://djangoproject.com) framework. It needs a [PostgreSQL](https://www.postgresql.org/) database, with the [Postgis](https://postgis.net/) extension enabled.
Here are the commands to install the required system dependencies.
=== "Debian"
```bash
sudo apt update
sudo apt install python3 python3-dev python3-venv virtualenv postgresql gcc postgis libpq-dev
```
=== "Arch Linux"
```bash
yay postgis extra/postgresql-libs
```
=== "OS X (with brew)"
```bash
brew install postgis
```
=== "Fedora"
```bash
sudo dnf install postgis libpq-devel make gcc python3-devel
```
### PostgreSQL
Depending on your system, you might need to create a postgres user, the database, and initialize postgres. Here's how:
```bash
createuser umap -U postgres
createdb umap -O umap -U postgres
psql umap -c "CREATE EXTENSION postgis" -Upostgres
```
## Getting started
Create a geo aware database. See the [GeoDjango docs](https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/) for backend installation.
### Creating a virtual environment
It is recommended to install python projects in a virtual environment to avoid mixing the project installation with your system dependencies. But it's not a requirement and it is up to you 🫣
```bash
python -m venv venv
source venv/bin/activate
```
Install dependencies and project
### Installing the dependencies
You can get all the project dependencies installed with the following command:
```bash
pip install umap-project
```
Create a default `local_settings` file
### Configuration
Create a default `local_settings.py` file, that you will modify with your setting.
```bash
wget https://raw.githubusercontent.com/umap-project/umap/master/umap/settings/local.py.sample -O local_settings.py
```
Reference it as env var:
```bash
export UMAP_SETTINGS=`pwd`/local_settings.py
```
Add database connection information in `local_settings.py`, for example
```python
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'umap',
}
}
```
Depending on your installation, you might need to change the USER that connects the database.
Depending on your installation, you might need to change the user that connects the database.
It should look like this:
```python
DATABASES = {
"default": {
"ENGINE": "django.contrib.gis.db.backends.postgis",
@ -45,19 +106,23 @@ It should look like this:
"USER": "postgres",
}
}
```
Add a `SECRET_KEY` in `local_settings.py` with a long random secret key
```python
SECRET_KEY = "a long and random secret key that must not be shared"
```
You can easily generate one with openssl:
You can easily generate one with [openssl](https://www.openssl.org/):
```bash
openssl rand -base64 32
```
uMap uses [python-social-auth](http://python-social-auth.readthedocs.org/) for user authentication. So you will need to configure it according to your
needs. For example
uMap uses [python-social-auth](http://python-social-auth.readthedocs.org/) for user authentication. So you will need to configure it according to your needs. For example
```python
AUTHENTICATION_BACKENDS = (
'social_auth.backends.contrib.github.GithubBackend',
'social_auth.backends.contrib.bitbucket.BitbucketBackend',
@ -70,6 +135,7 @@ needs. For example
BITBUCKET_CONSUMER_SECRET = 'zzz'
TWITTER_CONSUMER_KEY = "xxx"
TWITTER_CONSUMER_SECRET = "yyy"
```
Example of callback URL to use for setting up OAuth apps
@ -77,28 +143,31 @@ Example of callback URL to use for setting up OAuth apps
Adapt the `STATIC_ROOT` and `MEDIA_ROOT` to your local environment.
Create the tables
## Bootstrapping the database
Here are the commands you'll need to run to create the tables, collect the static files, etc.
```bash
# Create the database tables
umap migrate
Collect and compress the statics
# Collect and compress static files
umap collectstatic
umap compress
Create a superuser
# Create a super user
umap createsuperuser
Start the server
# Finally start the server
umap runserver 0.0.0.0:8000
```
## Search
## Configuring PostgreSQL search
UMap uses PostgreSQL tsvector for searching. In case your database is big, you
may want to add an index. For that, you should do so:
may want to add an index. For that, here are the SQL commands to run:
```SQL
# Create a basic search configuration
CREATE TEXT SEARCH CONFIGURATION umapdict (COPY=simple);
@ -109,11 +178,15 @@ may want to add an index. For that, you should do so:
# Now create the index
CREATE INDEX IF NOT EXISTS search_idx ON umap_map USING GIN(to_tsvector('umapdict', COALESCE(name, ''::character varying)::text), share_status);
```
And change `UMAP_SEARCH_CONFIGURATION = "umapdict"` in your settings.
And change your settings:
```python
UMAP_SEARCH_CONFIGURATION = "umapdict"
```
## Emails
## Configuring emails
UMap can send the anonymous edit link by email. For this to work, you need to
add email specific settings. See [Django](https://docs.djangoproject.com/en/4.2/topics/email/#smtp-backend)
@ -121,7 +194,7 @@ documentation.
In general, you'll need to add something like this in your local settings:
```
```python
FROM_EMAIL = "youradmin@email.org"
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_HOST = "smtp.provider.org"
@ -133,11 +206,11 @@ EMAIL_USE_TLS = True
EMAIL_USE_SSL = True
```
## Upgrading
## Upgrading your installation
Usually, for upgrading, you need those steps:
```
```bash
pip install umap-project --upgrade
umap migrate
umap collectstatic
@ -146,6 +219,6 @@ umap compress
Then you need to restart your python server, for example:
```
```bash
sudo systemctl restart uwsgi # or gunicorn, or…
```

144
docs/install_windows.md
View file

@ -1,144 +0,0 @@
# Installing uMap on Windows
The **good news** is that it is possible to run uMap server on Windows. However, it is recommended using uMap on a
Linux distribution as it will be easier to install, modify, and deploy. While the following steps have been tested on
Windows 7, they may work for other versions of Windows.
## 1. Prepare the Database
This assumes you've installed PostgreSQL.
- Create a database called "umap"
- Install PostGIS extension in it
##2. Create a directory and a Python virtual environment
This assumes you've installed Python (version 3.8+ 64-bit is a good choice) and virtualenv.
Open a Windows command window, and cd to a directory of your choice. You need to create a sub-directory but the name is
up to you (it doesn't need to be called "production"):
```
mkdir production
cd production
virtualenv venv
venv\Scripts\activate.bat
```
##3. Install GDAL for Python
It is really difficult to install GDAL the "standard" way since it requires compiling GDAL. Instead download an already
compiled pip-compatible wheel package file from
[Unofficial Windows Binaries for Python Extension Packages](https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal). Note
that cp38 refers to the Python version you are using, so make sure you select the one that matches your Python version
for download.
In the command window, install the downloaded wheel package:
`pip install GDAL-3.0.4-cp38-cp38-win_amd64.whl`
You can test the install from the Python command line. From the Windows command window invoke Python:
```
python
```
then enter some Python commands:
```python
>>> import gdal
>>> print(int(gdal.VersionInfo('VERSION_NUM')))
>>> exit()
```
##4. Install uMap
In the Windows command window:
```
mkdir static
mkdir data
pip install umap-project
```
***Windows Work-Around 1***
Setting the UMAP_SETTINGS environment variable doesn't seem to work on Windows, so put the file in umap's fall-back
location of \etc\umap\umap.conf :
```
mkdir \etc\umap
wget https://raw.githubusercontent.com/umap-project/umap/master/umap/settings/local.py.sample -O \etc\umap\umap.conf
```
Edit the umap.conf file:
***Windows Work-Around 2***
It might be possible to modify django's libgdal.py (umap installed django as one of its dependencies) to detect the
installed GDAL, but until then you can explicitly state the required paths.
Add the GDAL paths somewhere near the top of the umap.conf file (make sure the last part, "gdal300", is the name of the
GDAL DLL in its package dir):
```python
GDAL_LIBRARY_PATH = r'C:\temp\production\venv\Lib\site-packages\osgeo\gdal300'
GEOS_LIBRARY_PATH = r'C:\temp\production\venv\Lib\site-packages\osgeo\geos_c'
PROJ_LIB = r'C:\temp\production\venv\Lib\site-packages\osgeo\data\proj'
```
And while you're editing umap.conf, add the needed parameters to the DATABASES default object :
```python
DATABASES = {
'default': {
'ENGINE': 'django.contrib.gis.db.backends.postgis',
'NAME': 'umap',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'localhost',
'PORT': '5432'
}
}
```
And set umap's paths to where you've created the directories:
```python
STATIC_ROOT = '/temp/production/static'
MEDIA_ROOT = '/temp/production/data'
```
Now that the minimal configuration is done, you can do the django-ish portion of the umap install. In the Windows
command window:
```
umap migrate
umap collectstatic
umap createsuperuser
```
***Windows Work-Around 3***
Strangely, having the installed `umap.exe` is not enough. Some script tries to execute "umap" without the ".exe"
extension, so here's a hack to make that work:
```
copy venv\scripts\umap.exe venv\scripts\umap
```
##5. Run umap server
In the Windows command window:
```
umap runserver 127.0.0.1:8000
```
You should now be able to open a browser and go to http://127.0.0.1:8000
If you add some features to a new map and try to save them, you will likely see an error in the Windows command window
running the umap server. This error is a Python error related to doing
`os.remove(name)` on Windows:
```
File "c:\temp\test\venv\lib\site-packages\django\core\files\storage.py", line 303, in delete
os.remove(name)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
'C:\\temp\\production\\data\\datalayer\\1\\1\\layer-1.geojson'
```
***Windows Work-Around 4***
Edit `test\venv\lib\site-packages\django\core\files\storage.py`, and comment out lines 302 and 303:
```python
# else:
# os.remove(name)
```
Now adding features and saving should work. _Now here's the weird part._ Edit `storage.py` to restore it to it's
original state by removing the comment characters you put in. Save the changes, do some more feature editing and
saving in your browser. It still works! This may be due to file/directory locking by Windows.
##6. Installing for development
The previous sections describe the install procedure for running the released version of uMap "as-is". If you want to
modify uMap (and possibly contribute your changes back to the uMap team), have a look at [Contributing](contributing.md)

2
docs/settings.md
View file

@ -5,7 +5,7 @@ are doing.
The Django settings reference is here: https://docs.djangoproject.com/en/4.2/ref/settings/
Here are a few relevent settings for uMap.
Here are a few relevant settings for uMap.
## Usage

37
mkdocs.yml
View file

@ -1,15 +1,46 @@
site_name: uMap
site_description: uMap lets you create maps with OpenStreetMap layers in a minute and embed them in your site
repo_name: umap-project/umap
repo_url: https://github.com/umap-project/umap
nav:
- Home: index.md
- Installation: install.md
- Architecture:
- Frontend (JS): architecture/frontend.md
- Deployment:
- Ubuntu Server Howto: deploy/ubuntu.md
- Docker: deploy/docker.md
- Windows: deploy/windows.md
- Configuration: settings.md
- Administration: administration.md
- Contributing: contributing.md
- how-tos:
- Ubuntu from scratch: ubuntu.md
- Customize your uMap style: custom.md
- Install with Docker: docker.md
- Changelog: changelog.md
theme: readthedocs
theme:
name: material
palette:
- scheme: 'default'
media: '(prefers-color-scheme: light)'
toggle:
icon: 'material/lightbulb'
name: "Switch to dark mode"
- scheme: 'slate'
media: '(prefers-color-scheme: dark)'
primary: 'blue'
toggle:
icon: 'material/lightbulb-outline'
name: 'Switch to light mode'
features:
- navigation.sections
markdown_extensions:
- pymdownx.magiclink
- admonition
- pymdownx.superfences:
custom_fences:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
- pymdownx.tabbed:
alternate_style: true
combine_header_slug: true

1
pyproject.toml
View file

@ -48,6 +48,7 @@ dev = [
"ruff==0.1.6",
"djlint==1.31.0",
"mkdocs==1.5.2",
"mkdocs-material>=9.4.14,<10",
"vermin==1.5.2",
"pymdown-extensions==10.4",
"isort==5.12",