Skip to content

RenderingOsmDataUbuntu

dmorissette edited this page Jul 10, 2012 · 5 revisions

This page provides step by step instructions to setup a Ubuntu 10.10 LTS server from scratch to render OSM data with MapServer 6.x.

The following instructions assume that we start with a brand new Ubuntu 10.10 server with Apache installed but none of the MapServer related packages installed.

The setup will include:

Create working directory

  • All the steps that follow assume that the data and mapfiles will be installed in a directory called "osm-demo" in your home directory
  • Commands:
  mkdir ~/osm-demo                                                                                                                                                                                                                
  cd ~/osm-demo/                                                                                                                                                                                                                  

Install UbuntuGIS MapServer and PostGIS packages

  sudo apt-get install python-software-properties                                                                                                                                                                                 
  sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable                                                                                                                                                                        
  sudo apt-get update                                                                                                                                                                                                             
  sudo apt-get install cgi-mapserver mapserver-bin                                                                                                                                                                                
                                                                                                                                                                                                                                  
  sudo apt-get install postgresql                                                                                                                                                                                                 
  sudo apt-get install postgis postgresql-8.4-postgis                                                                                                                                                                             
  • Install other non-GIS packages that will be required later on:
  sudo apt-get install mercurial subversion git                                                                                                                                                                                      
  sudo apt-get install zip unzip                                                                                                                                                                                                  

Download OSM data

  • You can download data for your region of interest from http://download.geofabrik.de/osm/. The rest of these instructions assume that we work with the data for the state of Alabama.
  • Commands:
  cd ~/osm-demo/                                                                                                                                                                                                                  
  wget http://download.geofabrik.de/osm/north-america/us/alabama.osm.pbf                                                                                                                                                          

Install imposm, using virtualenv

  cd ~/osm-demo/                                                                                                                                                                                                                  
  sudo apt-get install build-essential python-dev protobuf-compiler \                                                                                                                                                             
                      libprotobuf-dev libtokyocabinet-dev python-psycopg2 \                                                                                                                                                       
                      libgeos-c1                                                                                                                                                                                                  
                                                                                                                                                                                                                                  
  sudo apt-get install python-virtualenv                                                                                                                                                                                          
  virtualenv venv                                                                                                                                                                                                                 
  source venv/bin/activate                                                                                                                                                                                                        
  • Install shapely speedups: (shapely 1.2.11 has bugs, do not use it)
  sudo apt-get install libgeos-dev                                                                                                                                                                                                
  pip install Shapely                                                                                                                                                                                                             
  pip install https://github.com/olt/shapely_speedups/tarball/master                                                                                                                                                              
  • Install imposm:
    Note that up until recently imposm did not create the generalized tables used by the mapserver-utils mapfile. Since version 2.3.0 this is not the case anymore, so there is no need
    to install a forked version of imposm as was the case in the first versions of this document
  pip install imposm                                                                                                                                                                                                              

Create database

  imposm-psqldb > create-db.sh                                                                                                                                                                                                    
  vi ./create-db.sh # cross check if all path are set properly                                                                                                                                                                    
... edit the following lines:                                                                                                                                                                                                     
-------------------8<--------------                                                                                                                                                                                               
psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql             # <- CHECK THIS PATH                                                                                                                         
psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql     # <- CHECK THIS PATH                                                                                                                         
------------------->8--------------                                                                                                                                                                                               
                                                                                                                                                                                                                                  
  sudo su postgres                                                                                                                                                                                                                
  sh ./create-db.sh                                                                                                                                                                                                               
  /etc/init.d/postgresql-8.4 restart                                                                                                                                                                                              
  exit                                                                                                                                                                                                                            

Load data using imposm

  cd ~/osm-demo/                                                                                                                                                                                                                  
  imposm --read alabama.osm.pbf                                                                                                                                                                                                   
  imposm --write --database osm --host localhost --user osm                                                                                                                                                                       
  (... if prompted for db password, the default is osm)                                                                                                                                                                           
  imposm  --optimize -d osm                                                                                                                                                                                                       
  • Note: multiple .osm.pbf files can be loaded in separate commands using the --merge-cache argument.

Install mapserver-utils mapfile generator

  • Website: http://code.google.com/p/mapserver-utils/
  • There are two branches of the mapserver-utils project that can be used, depending on your needs:
    • The trunk contains the latest and greatest developments
    • The 'imposm-dm-branch' is focused on Canada/US data, and contains supports for things such as US highway shields.
  • To use the 'imposm-dm-branch' from SVN:
  cd ~/osm-demo/                                                                                                                                                                                                                  
  svn checkout http://mapserver-utils.googlecode.com/svn/branches/imposm-dm-branch/ mapserver-utils-svn-imposm                                                                                                                    
  cd mapserver-utils-svn-imposm                                                                                                                                                                                                   
... no files to edit, the changes listed for trunk below are already included in the imposm-dm-branch                                                                                                                             
  • To use the trunk from SVN:
  cd ~/osm-demo/                                                                                                                                                                                                                  
  svn checkout http://mapserver-utils.googlecode.com/svn/trunk/ mapserver-utils-svn                                                                                                                                               
  cd mapserver-utils-svn                                                                                                                                                                                                          
                                                                                                                                                                                                                                  
... Update mapserver-utils's osmbase.map and Makefile as follows:                                                                                                                                                                 
  vi osm-base.map                                                                                                                                                                                                                 
-------------------8<------------------------                                                                                                                                                                                     
  WEB                                                                                                                                                                                                                             
...                                                                                                                                                                                                                               
    IMAGEPATH "/tmp/ms_tmp/"                                                                                                                                                                                                      
    IMAGEURL "/ms_tmp/"                                                                                                                                                                                                           
 END                                                                                                                                                                                                                              
...                                                                                                                                                                                                                               
-------------------->8-----------------------                                                                                                                                                                                     
                                                                                                                                                                                                                                  
  vi Makefile                                                                                                                                                                                                                     
-------------------8<------------------------                                                                                                                                                                                     
OSM_SRID=900913                                                                                                                                                                                                                   
OSM_UNITS=meters                                                                                                                                                                                                                  
OSM_EXTENT=-20000000 -20000000 20000000 20000000                                                                                                                                                                                  
...                                                                                                                                                                                                                               
STYLE=google                                                                                                                                                                                                                      
-------------------->8-----------------------                                                                                                                                                                                     
  • With trunk, the '''STYLE''' parameter possible values are the keys of the style_aliases hash, which can be found in the generate_style.py file. For example: google
  • Create MapServer temp dirs (required by mapserv CGI for testing with openlayers template)
  mkdir /tmp/ms_tmp                                                                                                                                                                                                               
  chmod 777 /tmp/ms_tmp                                                                                                                                                                                                           
  • Execute the mapserver-utils makefile to generate the mapfile. Note that the first time you run 'make' several large files will be downloaded (country boundaries, etc.). This will happen only the first time.
  cd mapserver-utils-svn                                                                                                                                                                                                          
  make                                                                                                                                                                                                                            
  • The 'make' command will have generated the mapfile based on the parameters in the generate_style.py script (either osm-outlined,google,usshields.map or osm-outlined,google.map depending on the branch that you chose above).

  • More information about tweaking the map styles is available at http://code.google.com/p/mapserver-utils/wiki/TweakingMapStylesImposm

  • Access your map online using MapServer's built-in template=openlayers mode:

http://yourserver.tld/cgi-bin/mapserv?map=/path/to/osm-demo/mapserver-utils-svn/osm-outlined,google,usshields.map&mode=browse&template=openlayers&layers=all                                                                      
or                                                                                                                                                                                                                                
http://yourserver.tld/cgi-bin/mapserv?map=/path/to/osm-demo/mapserver-utils-svn/osm-outlined,google.map&mode=browse&template=openlayers&layers=all                                                                                

Setup MapCache

  sudo apt-get install autoconf apache2-dev libpixman-1-dev libcurl4-openssl-dev libpng-dev libjpeg-dev libgdal-dev libgeos-dev                                                                                                   
  • Checkout and build source:
  git clone https://github.com/mapserver/mapcache mapcache-git                                                                                                                                             
  cd mapcache-git                                                                                                                                                                                                                 
  autoconf                                                                                                                                                                                                                        
  ./configure                                                                                                                                                                                                                     
  make                                                                                                                                                                                                                            
                                                                                                                                                                                                                                  
  sudo make install-module                                                                                                                                                                                                        
  sudo apache2ctl restart                                                                                                                                                                                                         
  mkdir ~/osm-demo/mapcache                                                                                                                                                                                                       
  cp ~/osm-demo/mapcache-git/mapcache.xml ~/osm-demo/mapcache/mapcache-osm.xml                                                                                                                                                    
  vi ~/osm-demo/mapcache/mapcache-osm.xml                                                                                                                                                                                         
... make required changes to template to make it work with our installation:                                                                                                                                                      
                                                                                                                                                                                                                                  
  <cache name="disk" type="disk">                                                                                                                                                                                                 
...                                                                                                                                                                                                                               
       <base>/opt/osm-demo/mapcache/cache</base>                                                                                                                                                                                  
                                                                                                                                                                                                                                  
...                                                                                                                                                                                                                               
   <source name="osm" type="wms">                                                                                                                                                                                                 
      <http>                                                                                                                                                                                                                      
         <url>http://localhost/cgi-bin/mapserv?</url>                                                                                                                                                                             
      </http>                                                                                                                                                                                                                     
                                                                                                                                                                                                                                  
...                                                                                                                                                                                                                               
            <MAP>/path/to/osm-demo/mapserver-utils-svn-imposm/osm-outlined,google,usshields.map</MAP>                                                                                                                             
... or ...                                                                                                                                                                                                                        
            <MAP>/path/to/osm-demo/mapserver-utils-svn/osm-google.map</MAP>                                                                                                                                                       
...                                                                                                                                                                                                                               
                                                                                                                                                                                                                                  
   <tileset name="osm">                                                                                                                                                                                                           
      <metadata>                                                                                                                                                                                                                  
         <title>OSM MapServer served map</title>                                                                                                                                                                                  
         <abstract>see http://trac.osgeo.org/mapserver/wiki/RenderingOsmDataUbuntu</abstract>                                                                                                                                     
      </metadata>                                                                                                                                                                                                                 
      <source>osm</source>                                                                                                                                                                                                        
      <cache>disk</cache>                                                                                                                                                                                                         
      <format>PNG</format>                                                                                                                                                                                                        
      <grid>g</grid>                                                                                                                                                                                                              
      <grid>WGS84</grid>                                                                                                                                                                                                          
      <metatile>5 5</metatile>                                                                                                                                                                                                    
      <expires>10000</expires>                                                                                                                                                                                                    
      <auto_expire>86400</auto_expire>                                                                                                                                                                                            
      <metabuffer>10</metabuffer>                                                                                                                                                                                                 
   </tileset>                                                                                                                                                                                                                     
  • Create 'cache' directory writable by Apache (www-data) user
  mkdir ~/osm-demo/mapcache/cache                                                                                                                                                                                                 
  sudo chown www-data ~/osm-demo/mapcache/cache/                                                                                                                                                                                  
  • Add directives to Apache config:
  sudo vi /etc/apache2/sites-available/default                                                                                                                                                                                    
... add the following lines to the end of the default VirtualHost (update the '/path/to/' directory name):                                                                                                                        
                                                                                                                                                                                                                                  
  <IfModule mapcache_module>                                                                                                                                                                                                      
    MapCacheAlias /mapcache "/path/to/osm-demo/mapcache/mapcache-osm.xml"                                                                                                                                                         
  </IfModule>                                                                                                                                                                                                                     
                                                                                                                                                                                                                                  
  • And restart apache for the changes to take effect:
  sudo apache2ctl restart                                                                                                                                                                                                         
Clone this wiki locally