Skip to content

Commit

Permalink
Fix up images
Browse files Browse the repository at this point in the history
  • Loading branch information
cguess committed Dec 14, 2023
1 parent 55b3b29 commit f0da324
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 4 deletions.
7 changes: 6 additions & 1 deletion lib/mosquito.rb
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,17 @@ def self.tweet_fields
# Get media from a URL and save to a temp folder set in the configuration under
# temp_storage_location
def self.retrieve_media(url)
return "" if url.nil?
return "" if !Mosquito.save_media

response = Typhoeus.get(url)

# Get the file extension if it's in the file
extension = url.split(".").last
begin
extension = url.split(".").last
rescue StandardError => e
debugger
end

# Do some basic checks so we just empty out if there's something weird in the file extension
# that could do some harm.
Expand Down
7 changes: 5 additions & 2 deletions lib/mosquito/scrapers/tweet_scraper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,11 @@ def parse(id)
video_preview_image = nil
video_file_type = nil

# Single image
images.concat(Mosquito.retrieve_media(doc.xpath("//a[contains(@class, 'still-image')][1]/href")))
# # Single image
# image_url = doc.xpath("//div[contains(@class, 'main-tweet')]/div/div/div/div/div/a[contains(@class, 'still-image')]/@href").first&.content
# images << Mosquito.retrieve_media("#{Capybara.app_host}#{image_url}") unless image_url.nil?

# debugger

# Slideshow
nodes = doc.xpath("//div[contains(@class, 'main-tweet')]/div/div/div[contains(@class, 'attachments')]/div[contains(@class, 'gallery-row')]/div/a/@href")
Expand Down
2 changes: 1 addition & 1 deletion test/user_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def test_that_a_user_has_correct_attributes
assert_equal "f_obermaier", user.username
assert_equal "Threema FPN4FKZE | PGP", user.location
assert user.description.include? "journalist"
assert_equal nil, user.url
assert_nil user.url
assert_kind_of Integer, user.followers_count
assert_kind_of Integer, user.following_count
assert_kind_of Integer, user.tweet_count
Expand Down

0 comments on commit f0da324

Please sign in to comment.