Skip to content

Files

Latest commit

c8bb82b · Jul 2, 2010

History

History
This branch is up to date with b/cookbooks:master.

hadoop

DESCRIPTION:

Installs Apache hadoop and sets up a basic distributed cluster per the quick start documentation.

REQUIREMENTS:

Platform:

Tested on Ubuntu 8.10, though should work on most Linux distributions, see hadoop.

Cookbooks:

Opscode cookbooks, github.com/opscode/cookbooks/tree/master:

  • java

ATTRIBUTES:

  • hadoop - Get a mirror from www.apache.org/dyn/closer.cgi/hadoop/core/.

  • hadoop - Specify the version of hadoop to install.

  • hadoop - Default userid of the hadoop user.

  • hadoop - Default group for the hadoop user.

  • hadoop - You will probably want to change this to match where Java is installed on your platform.

You may wish to add more attributes for tuning the configuration file templates.

USAGE:

This cookbook performs the tasks described in the Hadoop Quick Start to get the software installed. You should copy this to a site-cookbook and modify the templates to meet your requirements.

Once the recipe is run, the distributed filesystem can be formated using the script /usr/bin/hadoop.

sudo -u hadoop /usr/bin/hadoop namenode -format

You may need to set up SSH keys for hadoop management commands.

Note that this is not the ‘default’ config per se, so using the start-all.sh script won’t start the processes because the config files live elsewhere. For running various hadoop processes as services, we suggest runit. A sample ‘run’ script is provided. The HADOOP_LOG_DIR in the run script must exist for each process. These could be wrapped in a define.

  • datanode

  • jobtracker

  • namenode

  • tasktracker

1

hadoop.apache.org/core/docs/current/quickstart.html

LICENSE and AUTHOR:

Author

Joshua Timberman (<[email protected]>)

Copyright

2009, Opscode, Inc

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.