Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Download Support #12

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
96f9253
switched source download to git
ngtjah Dec 15, 2015
6da6d95
switched source download to git
ngtjah Dec 15, 2015
9a3e223
switched source download to git
ngtjah Dec 15, 2015
f2d5041
switched source download to git
ngtjah Dec 15, 2015
92da2f5
switched source download to git
ngtjah Dec 15, 2015
2f68b84
switched source download to git
ngtjah Dec 15, 2015
36f8bcc
switched source download to git
ngtjah Dec 15, 2015
e6dff7f
switched source download to git
ngtjah Dec 15, 2015
091959e
switched source download to git
ngtjah Dec 15, 2015
7f0aa76
switched source download to git
ngtjah Dec 15, 2015
5314785
switched source download to git
ngtjah Dec 15, 2015
1a37c79
switched source download to git
ngtjah Dec 15, 2015
3abef54
switched source download to git
ngtjah Dec 15, 2015
46d6710
switched source download to git
ngtjah Dec 15, 2015
ff970b9
switched source download to git
ngtjah Dec 15, 2015
9555c04
switched source download to git
ngtjah Dec 15, 2015
b817e77
switched source download to git
ngtjah Dec 15, 2015
d3f7e33
switched source download to git
ngtjah Dec 15, 2015
bf51c01
switched source download to git
ngtjah Dec 15, 2015
ca5e99d
switched source download to git
ngtjah Dec 15, 2015
0e3ff4d
switched source download to git
ngtjah Dec 15, 2015
0f0408e
switched source download to git
ngtjah Dec 15, 2015
64acb74
readme
ngtjah Dec 15, 2015
888989a
readme
ngtjah Dec 15, 2015
97c2156
readme
ngtjah Dec 15, 2015
73344c0
readme
ngtjah Dec 15, 2015
debdc85
readme
ngtjah Dec 15, 2015
01a3774
set autogen version correctly
ngtjah Dec 15, 2015
9aca2f9
nvm
ngtjah Dec 15, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
.idea/
.kitchen/
.kitchen.local.yml
65 changes: 65 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,35 @@ GEM
plist (~> 3.1.0)
pry (~> 0.9)
rest-client (>= 1.0.4, <= 1.6.7)
chef (11.16.2-x86-mingw32)
chef-zero (~> 2.1, >= 2.1.4)
diff-lcs (~> 1.2, >= 1.2.4)
erubis (~> 2.7)
ffi (~> 1.9)
ffi-yajl (~> 1.0)
highline (~> 1.6, >= 1.6.9)
mime-types (~> 1.16)
mixlib-authentication (~> 1.3)
mixlib-cli (~> 1.4)
mixlib-config (~> 2.0)
mixlib-log (~> 1.3)
mixlib-shellout (~> 1.4)
net-ssh (~> 2.6)
net-ssh-multi (~> 1.1)
ohai (~> 7.4)
plist (~> 3.1.0)
pry (~> 0.9)
rest-client (>= 1.0.4, <= 1.6.7)
win32-api (= 1.5.1)
win32-dir (= 0.4.5)
win32-event (= 0.6.1)
win32-mmap (= 0.4.0)
win32-mutex (= 0.4.1)
win32-process (= 0.7.3)
win32-service (= 0.8.2)
windows-api (= 0.4.2)
windows-pr (= 1.2.2)
wmi-lite (~> 1.0)
chef-zero (2.2)
hashie (~> 2.0)
json
Expand All @@ -39,6 +68,7 @@ GEM
net-ssh
ohai
ffi (1.9.5)
ffi (1.9.5-x86-mingw32)
ffi-yajl (1.1.0)
ffi (~> 1.5)
libyajl2 (~> 1.0)
Expand All @@ -53,6 +83,8 @@ GEM
formatador (0.2.5)
gherkin (2.12.2)
multi_json (~> 1.3)
gherkin (2.12.2-x86-mingw32)
multi_json (~> 1.3)
guard (2.6.1)
formatador (>= 0.2.4)
listen (~> 2.7)
Expand All @@ -68,6 +100,7 @@ GEM
hashie (2.1.2)
highline (1.6.21)
hitimes (1.2.2)
hitimes (1.2.2-x86-mingw32)
ipaddress (0.8.0)
json (1.8.1)
kitchen-docker (1.5.0)
Expand Down Expand Up @@ -97,6 +130,9 @@ GEM
mixlib-config (2.1.0)
mixlib-log (1.6.0)
mixlib-shellout (1.4.0)
mixlib-shellout (1.4.0-x86-mingw32)
win32-process (~> 0.7.1)
windows-pr (~> 1.2.2)
multi_json (1.10.1)
net-scp (1.2.1)
net-ssh (>= 2.6.5)
Expand All @@ -108,6 +144,8 @@ GEM
net-ssh-gateway (>= 1.2.0)
nokogiri (1.6.3.1)
mini_portile (= 0.6.0)
nokogiri (1.6.3.1-x86-mingw32)
mini_portile (= 0.6.0)
ohai (7.4.0)
ffi (~> 1.9)
ffi-yajl (~> 1.0)
Expand All @@ -125,6 +163,11 @@ GEM
coderay (~> 1.1.0)
method_source (~> 0.8.1)
slop (~> 3.4)
pry (0.10.1-x86-mingw32)
coderay (~> 1.1.0)
method_source (~> 0.8.1)
slop (~> 3.4)
win32console (~> 1.3)
rack (1.5.2)
rake (10.3.2)
rb-fsevent (0.9.4)
Expand Down Expand Up @@ -165,11 +208,33 @@ GEM
hitimes
treetop (1.5.3)
polyglot (~> 0.3)
win32-api (1.5.1-x86-mingw32)
win32-dir (0.4.5)
ffi (>= 1.0.0)
win32-event (0.6.1)
win32-ipc (>= 0.6.0)
win32-ipc (0.6.6)
ffi
win32-mmap (0.4.0)
ffi
win32-mutex (0.4.1)
win32-ipc (>= 0.6.0)
win32-process (0.7.3)
ffi (>= 1.0.0)
win32-service (0.8.2)
ffi
win32console (1.3.2-x86-mingw32)
windows-api (0.4.2)
win32-api (>= 1.4.5)
windows-pr (1.2.2)
win32-api (>= 1.4.5)
windows-api (>= 0.3.0)
wmi-lite (1.0.0)
yajl-ruby (1.2.1)

PLATFORMS
ruby
x86-mingw32

DEPENDENCIES
chefspec
Expand Down
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,27 @@ Usage

```ruby
override_attributes(
's3fs-fuse' => {
's3fs_fuse' => {
:s3_key => 'key',
:s3_secret => 'secret',
:version => 'v1.79'
:mounts => [
{:bucket => 'my-bucket', :path => '/mount/path', :tmp_store => '/tmp/cache'}
],
:bluepill => true,
:maxmemory => 50
}
)
```

Multiple buckets can be mounted (which is why the `:mounts` attribute is an Array
Multiple buckets can be mounted (which is why the ```:mounts``` attribute is an Array
of Hashes). Bluepill monitoring is optional and the maxmemory allows bluepill
to kill off and remount any s3 mounts that misbehave.

The value of ```node[:s3fs_fuse][:version]``` will be used to download the specific tag
from the [s3fs-fuse github](https://github.com/s3fs-fuse/s3fs-fuse) for versions 1.74 and greater.

Note: Bluepill should be considered for any s3fs built bucket that has large number
of entries within directories. Running an `ls` on these directories will cause the
of entries within directories. Running an ```ls``` on these directories will cause the
mount's memory to balloon. Bluepill will happily watch for this (ballooning memory)
and remount the bucket.
2 changes: 1 addition & 1 deletion attributes/default.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
default[:s3fs_fuse][:s3_url] = 'https://s3.amazonaws.com'
default[:s3fs_fuse][:s3_key] = ''
default[:s3fs_fuse][:s3_secret] = ''
default[:s3fs_fuse][:version] = '1.61'
default[:s3fs_fuse][:version] = 'v1.79'
default[:s3fs_fuse][:no_upload] = false
default[:s3fs_fuse][:mounts] = []
default[:s3fs_fuse][:bluepill] = false
Expand Down
46 changes: 36 additions & 10 deletions recipes/install.rb
Original file line number Diff line number Diff line change
Expand Up @@ -44,23 +44,49 @@
package prereq_name
end

s3fs_version = node[:s3fs_fuse][:version]
source_url = "http://s3fs.googlecode.com/files/s3fs-#{s3fs_version}.tar.gz"
s3fs_git_tag = node[:s3fs_fuse][:version]
s3fs_version = node[:s3fs_fuse][:version].tr('^0-9.','')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a little more background why this change was needed?


remote_file "/tmp/s3fs-#{s3fs_version}.tar.gz" do
puts s3fs_version.to_f.to_s

if s3fs_version.to_f >= 1.74
source_url = "https://github.com/s3fs-fuse/s3fs-fuse/tarball/tags/#{s3fs_git_tag}"
else
source_url = "http://s3fs.googlecode.com/files/s3fs-#{s3fs_git_tag}.tar.gz"
end


remote_file "/tmp/s3fs-#{s3fs_git_tag}.tar.gz" do
source source_url
action :create_if_missing
end

directory "/tmp/s3fs-#{s3fs_git_tag}" do
mode 0755
action :create
end


bash "compile_and_install_s3fs" do
cwd '/tmp'
code <<-EOH
tar -xzf s3fs-#{s3fs_version}.tar.gz
cd s3fs-#{s3fs_version}
#{'export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig' if node.platform_family == 'rhel'}
./configure --prefix=/usr/local
make && make install
EOH
if s3fs_version.to_f >= 1.74 # Added ./autogen.sh
code <<-EOH
tar -xzf s3fs-#{s3fs_git_tag}.tar.gz -C s3fs-#{s3fs_git_tag} --strip-components 1
cd s3fs-#{s3fs_git_tag}
#{'export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig' if node.platform_family == 'rhel'}
./autogen.sh
./configure --prefix=/usr/local
make && make install
EOH
else
code <<-EOH
tar -xzf s3fs-#{s3fs_git_tag}.tar.gz -C s3fs-#{s3fs_git_tag} --strip-components 1
cd s3fs-#{s3fs_git_tag}
#{'export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig' if node.platform_family == 'rhel'}
./configure --prefix=/usr/local
make && make install
EOH
end
not_if do
begin
%x{s3fs --version}.to_s.split("\n").first.to_s.split.last == s3fs_version.to_s
Expand Down