-
Notifications
You must be signed in to change notification settings - Fork 6
Home
Currently, we're in the process of installing a Storm network on a local mini cluster at the university. The cluster consists of a handful nodes all running Ubuntu 12.04 with OpenJDK 6. Storm is installed and run by a dedicated user. It requires the following things to work:
- ZooKeeper mini-cluster (only nimbus will run a ZooKeeper instance in our setup!)
- ZeroMQ
- JZMQ (java bindings for ZeroMQ)
- Supervisor for easy recovery of both ZooKeeper and Storm. We use daemontools for simplicity.
- Configuration
- Log in to target host.
- Download and run the script, executing all phases and setting nimbus.
$ wget https://raw.github.com/maltoe/storm-install/master/storm_install.sh \ && chmod +x storm_install.sh \ && storm_install.sh all <nimbus> <installdir>
Alternatively, you could probably run the script via ssh directly.
The JZMQ fork that should be used according to the storm wiki fails to compile with the more recent autoconf and libtools versions shipped with Ubuntu 12.04. Thus, I used the upstream version and in the script calibrated the commit to e2dd65. Seems to work.
- If all went fine (read the log!), execute start.sh to run storm.
$ <installdir>/start.sh
For administration of the storm cluster, you can run storm ui on Nimbus, for example:
[ssh to nimbus] $ cd <installdir>/storm/storm-0.8.1 $ bin/storm ui [point your browser to http://<nimbus>:8080/]
For now, I did not provide a script to run the UI in supervisory mode (i.e., using daemontools), although this is stated on the Storm website. However, I could not see why the UI should fail at all, as well as I did not want the UI to run indefinitely. If you want to run the UI continuously, consider using screen or nohup again.