Test Suite

Location

The MapServer test suite is called msautotest and is hosted in a separate repository at https://github.com/mapserver/msautotest

branching convention

Test suites are versionned, under branch names that match the mapserver branch they correspond to. I.e, the testsuite for the 6.2 mapserver version is located in the branch-6-2 msautotest branch

submodule inside the mapserver repository

The msautotest repository is included as a git submodule in the main mapserver repository. A submodule references a specific commit in the msautotest history, and must therefore be updated whenever changes are applied to the msautotest suite itself. The procedure to accomplish this e.g. in the branch-6-2 branch is:

cd mapserver-srcdir
git checkout branch-6-2
git submodule update
cd msautotest
git checkout branch-6-2
git pull origin branch-6-2  #update the msautotest to the latest version for the current branch
cd ..
git add msautotest
git commit -m "updated msautotest to latest 6.2 revision"
git push origin branch-6-2

Reference Platform

As the output of the tests largely depend on the compiled-in MapServer options and more subtly to the actual platform version, cpu type, compiler version or compiler flags, we require expected results to have been generated by a mapserver binary configured strictly like this:

• For "textual" tests (capabilities, queries, gml, etc...):

• GIS dependencies coming from the ubuntu-gis ppa. The full list of dependencies to install can be seen by following the before_install steps of https://github.com/mapserver/mapserver/blob/branch-6-2/.travis.yml

• mapserver ./configure'd with --with-gd --with-postgis --with-wmsclient --with-proj --with-wfsclient --with-kml --with-cairo --with-wcs --with-sos --with-geos --with-gdal --with-ogr --with-wfs

• For image tests (getmap, shp2img, etc..):

• platform: Ubuntu Server 12.04, 32 bits (i386)

• GIS dependencies coming from the ubuntu-gis ppa. The full list of dependencies to install can be seen by following the before_install steps of https://github.com/mapserver/mapserver/blob/branch-6-2/.travis.yml

• mapserver ./configure'd with --with-gd --with-postgis --with-wmsclient --with-proj --with-wfsclient --with-kml --with-cairo --with-wcs --with-sos --with-geos --with-gdal --with-ogr --with-wfs

Reference platform VM

TODO Provide a platform where devs can update or add tests that conform to the reference platform.

Running the tests

Some tests require a postgis database. At the root level of the msautotest repository is a bash script that will create and populate an "msautotest" database. Beware that this script will drop any previously present "msautotest" database. Specific test suites can be run manually by cd'ing into a subdirectory and running ./run-test.py. We currently maintain the following test suites:

• gdal : for gdal specific raster manipulation operations
• wxs : for ogc test cases
• query : for mapserver mode=map query tests
• renderers : for testing rendering output of the different output drivers
• misc : other unclassifiable tests

There is also a php test suite that requires the presence of phpunit, the installation of which can also be found in our .travis.yml before_install steps

Continuous integration

We rely on travis-ci to run our test suite, the results of which can be seen at http://travis-ci.org/#!/mapserver/mapserver/builds . The test suite is automatically run each time new code gets committed to the master or branch-6-2 branches. As of Sept. 20 2012, all tests pass successfully. It is the responsibility of the developer committing code to ensure that his changes has not broken any tests. The test suite is also run when pull requests are opened on our issue tracker, giving instant feedback on wether the proposed changes have unwanted side-effects.