Because I wanted to become a responsible developer, and have a complete measuring and monitoring solution for my own projects, I decided to cook up a Playbook in Ansible, that installs the following tools and its dependencies:
- StatsD
- Graphite
- Sentry
This provides you with the capabilities to measure anything you want in your application and otherwise, collect it with StatsD and store and visualize it with Graphite. As an added bonus, Sentry provides you with a way to collect and monitor the exceptions that occur in your applications.
Note that this installs the aforementioned tools with a production-ready configuration. So no SQLite and silly portnumbers.
For a write-up on why and how I created this, have a look here: http://dandydev.net/blog/statserver-setting-up-monitoring-and-statistics-with-ansible
For a guide on how to do meaningful measurements in your applications, have a look here: http://matt.aimonetti.net/posts/2013/06/26/practical-guide-to-graphite-monitoring/
- PostgreSQL database
- Nginx webserver/reverse proxy
- Python, Pip & VirtualEnv
- NodeJS
- Memcached
- The 3 core Graphite components:
- StatsD
- The latest stable version of Sentry and all of its Python dependencies
If you want to install this on a real server, you'll only need Ansible. Instructions for installing Ansible, can be found on their website, but I'll sum up here how I'd recommend installing Ansible:
- Linux: Use PIP to install Ansible by running
sudo pip install ansible
. - OS X: Install Ansible with Homebrew by running
brew install ansible
. Installation using Homebrew PIP (you're not using OS X's default Python, are you?) doesn't work because of hardcoded paths. - Windows: Please get yourself a proper development machine. (Windows isn't supported)
If you want to try Statserver out locally, on a virtual machine, you'll need Vagrant and a Virtual Machine provider of choice (VirtualBox is free and works out of the box with Vagrant).
After meeting the requirements, clone the repo:
$ git clone https://github.com/DandyDev/statserver
$ cd /path/to/statserver
You should now edit the statserver.yml
file and modify a few variables:
graphite > server
: The hostname of the URL on which you want to access Graphite. For running a local VM, graphite.local is fine. In production, this could be something like graphite.mystatserver.com.db_graphite > password
: a secure password for the Graphite databasedb_sentry > password
: a secure password for the Sentry databasesuperuser_sentry > email, username, password
: details of the Sentry superuser account that will automatically be createdsentry > server
: The hostname of the URL on which you want to access Sentry. For running a local VM, sentry.local is fine. In production, this could be something like sentry.mystatserver.com.sentry > url
: This should be set to the value you set for thesentry > server
variable, prefixed with the desired protocol (http or https)
Now, for creating a VM locally to try this out, you can do the following:
$ echo "graphite.local" >> /etc/hosts
$ echo "sentry.local" >> /etc/hosts
$ vagrant up
For provisioning a remote server with Sentry, StatsD and Graphite, do the following:
# replace mystatserver.com with the domainname or IP address on which your server can be reached.
$ echo "mystatserver.com" >> myinventory
$ ansible-playbook statserver.yml -i myinventory
Within 5 - 10 minutes or so, you should be able to access Graphite and Sentry on the specified url's.
If you're happy with the results, and want something to measure, you can start by adding Diamond to all your servers. This will send all kinds of nifty system metrics to your statserver. Be sure to configure it to send its metrics to StatsD instead of Graphite, for less overhead.
By default, the Vagrant box runs Ubuntu 12.04, but the playbook supports Debian 7 and CentOS 6.4 as well! To try those out locally, uncomment the appropriate lines in the Vagrantfile and comment out the Debian lines. For remote servers, you don't have to change anything.
To create a superuser for Graphite, log in as root, or turn to root on your server, and issue the following command:
export PYTHONPATH=/opt/graphite/webapp/; /opt/graphite/bin/python /opt/graphite/bin/django-admin.py createsuperuser --settings=graphite.settings
- This hasn't been tested on other Providers than VirtualBox yet
- On CentOS, the firewall is completely closed by default (at least on the box I tried it with). So you have to manually open the relevant port (80) using
iptables
, or if you're not concerned about security (because you're running it locally through Vagrant), you can always flush the firewall withiptables -F
.
If you have any suggestions, feel free to create an issue here on Github and/or fork this repo, make changes and submit a pull request!