Skip to content

ContenCrawler is an application that allows its users to retrieve the most popular articles from any website using action-cable in rails 5 (https://contentcrawler.herokuapp.com/)

Notifications You must be signed in to change notification settings

SimonSomlai/contentcrawler

Repository files navigation

Coach App

ContenCrawler is an application that allows its users to retrieve the most popular articles from any website (measured by total social shares and comments). Might be down on occasion since it's running on free dynos.

User demo login: [email protected] & password (Yeah, that's 'demon' instead of 'demo'. Typo :/)

Tech Stack

  • Front: HTML, CSS, JS (+EventSource)
  • Back: Ruby & RoR
  • Gems: Typhoeus, watir, nokogiri, social_shares, phantomjs, activerecord-import, devise, open_uri, magic
  • Db: Postgresql

Pics Or It Didn't Happen

Home Page

Website Index

Website Show

Show Me Da Magic

Most of the cool stuff is inside app/helpers/scraper_helper.rb

Explainer Video (click it)

IMAGE ALT TEXT HERE

TO DO

  • Refactor messy javascript
  • Implement ip rotation
  • Find solution for facebook rape rate limiting
  • Maybe try to actually market it a bit?

Made with <3 by TrueTech

About

ContenCrawler is an application that allows its users to retrieve the most popular articles from any website using action-cable in rails 5 (https://contentcrawler.herokuapp.com/)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published