-
Notifications
You must be signed in to change notification settings - Fork 132
Building a parcel
Building a parcel is a relatively simple affair - put all your files and metadata in a directory and then tar it up.
As discussed elsewhere, the internal layout of a parcel is up to you, beyond the meta directory. As an example, here is how the GPLExtras (LZO) plugin parcel looks:
meta/
meta/parcel.json
meta/gplextras_env.sh
lib/
lib/hadoop/
lib/hadoop/lib/
lib/hadoop/lib/hadoop-lzo.jar -> hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/hadoop/lib/COPYING.hadoop-lzo
lib/hadoop/lib/native/
lib/hadoop/lib/native/libgplcompression.a
lib/hadoop/lib/native/libgplcompression.la
lib/hadoop/lib/native/libgplcompression.lai
lib/hadoop/lib/native/libgplcompression.so.0.0.0
lib/hadoop/lib/native/libgplcompression.so.0
lib/hadoop/lib/native/libgplcompression.so
lib/hadoop/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/hadoop-0.20-mapreduce/
lib/hadoop-0.20-mapreduce/lib/
lib/hadoop-0.20-mapreduce/lib/hadoop-lzo.jar -> ../../hadoop/lib/hadoop-lzo.jar
lib/hadoop-0.20-mapreduce/lib/native/
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.a -> ../../../hadoop/lib/native/libgplcompression.a
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.la -> ../../../hadoop/lib/native/libgplcompression.la
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.lai -> ../../../hadoop/lib/native/libgplcompression.lai
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so.0.0.0 -> ../../../hadoop/lib/native/libgplcompression.so.0.0.0
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so.0 -> ../../../hadoop/lib/native/libgplcompression.so.0
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so -> ../../../hadoop/lib/native/libgplcompression.so
lib/hadoop-0.20-mapreduce/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar -> ../../hadoop/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/impala/
lib/impala/lib/
lib/impala/lib/COPYING.impala-lzo
lib/impala/lib/libimpalalzo.soFor a given parcel with a name and a version, the top level directory must be named name-version. So, for our GPLExtras example, the name is GPLEXTRAS and the version is 5.0.0-gplextras5b2.p0.32, so the directory name must be GPLEXTRAS-5.0.0-gplextras5b2.p0.32. (Note that you don't have to use as complicated version strings as we do.)
Creating a parcel file from the directory is a simple exercise of running the tar command. As with directory naming, parcel filenames must follow a specific format: [name]-[version]-[distro suffix].parcel. The list of valid distro suffixes is available here.
To continue our GPLExtras example:
$ tar zcvf GPLEXTRAS-5.0.0-gplextras5b2.p0.32-el6.parcel GPLEXTRAS-5.0.0-gplextras5b2.p0.32/ --owner=root --group=rootWe strongly recommend that file ownership be normalised to root when building the parcels. This ensures that no file is accidentally put in a situation where it could be modified or replaced by an unprivileged user on an end-user machine. CM attempts to normalise ownership when unpacking parcels, but it's generally good practice to do it at build time too.
To help make sure that the parcels you build are correct, you should run our validation tool, available in this git repo, against your parcels. The tool offers different granularities of operation - it can validate individual json files, a parcel directory, or a parcel file - so you can use it throughout the development process.
$ ./validator -p parcel.json$ ./validator -a alternatives.json$ ./validator -r permissions.json$ ./validator -d CDH-5.0.0-0.cdh5b2.p0.283/$ ./validator -f CDH-5.0.0-0.cdh5b2.p0.283-el6.parcelAs discussed in the section on the environment script, connecting a plugin parcel to a service requires the plugin parcel's environment script to append to specific environment variables that allow the service to discover the plugin. The specific variables to use vary, depending on which server the plugin is for.
This table lists the relevant variables for the various CDH services.