Skip to content

Gluster4

Jump to bottom
cnnaik edited this page May 29, 2018 · 4 revisions

Building GlusterFS

Below versions of GlusterFS are available in respective distributions at the time of creation of these build instructions:

  • Ubuntu 16.04 - 3.7.6
  • Ubuntu 18.04 - 3.13.2

The instructions provided below specify the steps to build GlusterFS 4.0.2 on Linux on IBM Z for following distributions:

  • RHEL (7.3, 7.4, 7.5)
  • SLES (12 SP2, 12 SP3)
  • Ubuntu (16.04, 18.04)

General Note:

  • When following the steps below please use a super user unless otherwise specified
  • A directory /<source root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Building and Installing GlusterFS

Step 1: Install the dependencies

  • RHEL (7.3, 7.4, 7.5)

    yum install wget git make gcc-c++ libaio-devel boost-devel expat-devel autoconf autoheader automake libtool flex bison openssl-devel libacl-devel sqlite-devel libxml2-devel python-devel python attr yajl nfs-utils xfsprogs popt-static sysvinit-tools psmisc libibverbs-devel librdmacm-devel readline-devel lvm2-devel glib2-devel fuse-devel bc

  • SLES (12 SP2, 12 SP3)

    zypper install wget which git make gcc-c++ libaio-devel boost-devel autoconf automake cmake libtool flex bison lvm2-devel libacl-devel python-devel python attr xfsprogs sysvinit-tools psmisc bc libopenssl-devel libxml2-devel sqlite3 sqlite3-devel popt-devel nfs-utils libyajl2 python-xml net-tools

  • Ubuntu (16.04)

    apt-get update
apt-get install make automake autoconf libtool flex bison pkg-config libssl-dev libxml2-dev python-dev libaio-dev libibverbs-dev librdmacm-dev libreadline-dev liblvm2-dev libglib2.0-dev liburcu-dev libcmocka-dev libsqlite3-dev libacl1-dev wget tar dbench git xfsprogs attr nfs-common yajl-tools sqlite3 libxml2-utils thin-provisioning-tools bc

  • Ubuntu (18.04)

    apt-get update
apt-get install make automake autoconf libtool flex bison pkg-config libssl1.0-dev libxml2-dev python-dev libaio-dev libibverbs-dev librdmacm-dev libreadline-dev liblvm2-dev libglib2.0-dev liburcu-dev libcmocka-dev libsqlite3-dev libacl1-dev wget tar dbench git xfsprogs attr nfs-common yajl-tools sqlite3 libxml2-utils thin-provisioning-tools bc

  • Install other dependencies

    • Building URCU (ONLY for RHEL and SLES)

      cd /<source_root>/
git clone git://git.liburcu.org/userspace-rcu.git
cd userspace-rcu
./bootstrap 
./configure
make
make install
ldconfig

    • Building thin-provisioning-tools (ONLY for RHEL)

      cd /<source_root>/
git clone https://github.com/jthornber/thin-provisioning-tools
cd thin-provisioning-tools
autoreconf
./configure
make
make install

Step 2: Download , Configure and Build GlusterFS

  • Download and configure source

    cd /<source_root>/
git clone https://github.com/gluster/glusterfs
cd glusterfs
git checkout v4.0.2
./autogen.sh
./configure --enable-gnfs			# for RHEL and Ubuntu
./configure --enable-gnfs --disable-events	# for SLES

  • Update rculist-extra.h (Only for RHEL)

    rm contrib/userspace-rcu/rculist-extra.h
cp /usr/local/include/urcu/rculist.h contrib/userspace-rcu/rculist-extra.h

  • Make Changes to the file /<source_root>/glusterfs/xlators/performance/io-threads/src/io-threads.h (Only for Ubuntu & SLES)

    @@ -37,7 +37,7 @@ struct iot_conf;
#define IOT_MAX_THREADS         64


-#define IOT_THREAD_STACK_SIZE   ((size_t)(256*1024))
+#define IOT_THREAD_STACK_SIZE   ((size_t)(512*1024))


typedef enum {

  • Build and install

    make
make install

  • Set environment variables

    export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
ldconfig

Step 3: Run the test cases (optional)

  • Building xfsprogs

    • Install Dependencies
      • For Ubuntu 
        apt-get install gettext libblkid-dev uuid-dev
      • For RHEL 
        yum install gettext libblkid-devel uuid-devel
      • For SLES 
        zypper install libblkid-devel uuid-devel gettext-tools
    • Build xfsprogs 
      cd /<source_root>/
wget -c https://mirrors.edge.kernel.org/pub/linux/utils/fs/xfs/xfsprogs/xfsprogs-4.15.1.tar.gz
tar -xvf xfsprogs-4.15.1.tar.gz
cd xfsprogs-4.15.1 
make
make install

  • Building DBENCH (Only for RHEL and SLES)

    cd /<source_root>/
git clone https://github.com/sahlberg/dbench
cd dbench
./autogen.sh
./configure
make
make install

  • Building yajl (Only for SLES)

    cd /<source_root>/
git clone https://github.com/lloyd/yajl
cd yajl
git checkout 2.1.0
mkdir build
cd build/
cmake ..
make
export PATH=$PATH:$PWD/yajl-2.1.0/bin

  • Make Changes to the file /<source_root>/glusterfs/tests/basic/distribute/throttle-rebal.t

    @@ -17,7 +17,7 @@ function set_throttle {
}
  
#Determine number of cores
-cores=$(cat /proc/cpuinfo | grep processor | wc -l)
+cores=$(cat /proc/cpuinfo | grep -w processor | wc -l)
if [ "$cores" == "" ]; then
  echo "Could not get number of cores available"
fi

  • Make Changes to the file /<source_root>/glusterfs/tests/basic/mount-nfs-auth.t

    @@ -15,16 +15,17 @@ TEST glusterd
TEST pidof glusterd
TEST $CLI volume info

-H0IP=$(ip addr show |grep -w inet |grep -v 127.0.0.1|awk '{ print $2 }'| cut -d "/" -f 1)
+#H0IP=$(ip addr show |grep -w inet |grep -v 127.0.0.1|awk '{ print $2 }'| cut -d "/" -f 1)
+H0IP=$(hostname -i | awk '{ print $1 }')
 H0IP6=$(host $HOSTNAME | grep IPv6 | awk '{print $NF}')

# Export variables for allow & deny
-EXPORT_ALLOW="/$V0 $H0(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
-EXPORT_ALLOW_SLASH="/$V0/ $H0(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
+EXPORT_ALLOW="/$V0 $H0IP(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
+EXPORT_ALLOW_SLASH="/$V0/ $H0IP(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
 EXPORT_DENY="/$V0 1.2.3.4(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"

# Netgroup variables for allow & deny
-NETGROUP_ALLOW="ngtop ng1000\nng1000 ng999\nng999 ng1\nng1 ng2\nng2 ($H0,,)"
+NETGROUP_ALLOW="ngtop ng1000\nng1000 ng999\nng999 ng1\nng1 ng2\nng2 ($H0IP,,)"
NETGROUP_DENY="ngtop ng1000\nng1000 ng999\nng999 ng1\nng1 ng2\nng2 (1.2.3.4,,)"

V0L1="$V0/L1"
@@ -32,8 +33,8 @@ V0L2="$V0L1/L2"
V0L3="$V0L2/L3"

# Other variations for allow & deny
-EXPORT_ALLOW_RO="/$V0 $H0(sec=sys,ro,anonuid=0) @ngtop(sec=sys,ro,anonuid=0)"
-EXPORT_ALLOW_L1="/$V0L1 $H0(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
+EXPORT_ALLOW_RO="/$V0 $H0IP(sec=sys,ro,anonuid=0) @ngtop(sec=sys,ro,anonuid=0)"
+EXPORT_ALLOW_L1="/$V0L1 $H0IP(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"
EXPORT_WILDCARD="/$V0 *(sec=sys,rw,anonuid=0) @ngtop(sec=sys,rw,anonuid=0)"

function build_dirs () {

  • Make Changes to the file /<source_root>/glusterfs/tests/basic/tier/frequency-counters.t

    @@ -6,8 +6,8 @@
NUM_BRICKS=3
-DEMOTE_FREQ=10
-PROMOTE_FREQ=10
+DEMOTE_FREQ=35
+PROMOTE_FREQ=35

NUM_FILES=5
TEST_DIR=test

  • Run the test cases

    cd /<source_root>/glusterfs
./run-tests.sh

Note:

  • If any of the tests fail with error volume create: patchy: failed: Host is not in 'Peer in Cluster' state, make changes to file /etc/hosts by copying the host details entry that is present in the beginning to the section IPv6 capable hosts and replace 0.0.0.0 with the IP address of the host. It should be similar to : 0.0.0.0 .xyz.com hostname

  • If you want to use the kernel NFS server or client on a Gluster storage server, make sure to disable the Gluster NFS server:

    gluster volume set <VOLUME> nfs.disable true

  • For geo-replication test case : Password-less SSH login has to be set up. Refer the Setting Up the Environment for Geo-replication in the Documentation

  • Additional information on GlusterFS test can be found here.

Step 4: Run the GlusterFS Daemon (Optional)

glusterd

Step 5: Testing the GlusterFS volume (Optional)

Refer Documentation for instructions.

Reference

https://www.gluster.org/
https://github.com/gluster/glusterfs
http://docs.gluster.org/en/latest/Developer-guide/Building-GlusterFS/

Clone this wiki locally