Twitalyse

Twitalyse: Analyse your tweets. NOW!

WARNING: pre-experimental, not usable right now.

Resources

First deployement on Google AppEngine:

twitalyse beta on appengine

Dev Doc

Installing

cd twitter-poc
lein deps
lein swank

twitter-poc

Checkup to verify that you can connect correctly to Twitter locally.

Running

• Go to the twitter-poc directory and start a swank server:

cd twitter-poc
lein deps
lein swank

• Setup the security:
  • the project needs the file twitalyse/twitter-poc/src/twitter_poc/secret.clj
  • inside you must put (where xxx and yyy are the consumer-secret and the oauth-access-token):

(ns twitter-poc.secret)

;;; Where the secret keys are stored.
;;; This file MUST NOT be pushed, it's in .gitignore

(def consumer-secret "xxx")
(def oauth-access-token-secret "yyy")

• In Emacs Slime:
  • M-x slime-connect
  • Load the core:

user> (use :reload 'twitalyse.twitter)
nil

• First query:

user> (first (map :text (latest-tweets 1)))
"Reviens d'un petit tour en ULM au-dessus du Mont St. Michel #cool"

• That’s it !

GAE developpement

GAE dev deploy

• Start swank lein swank
• Connect in Emacs Slime M-x slime-connect
• In the repl:

(require '[appengine-magic.core :as ae])

(use '[twitalyse.core])

(ae/serve twitalyse-app)

• You should see your app in http://localhost:8080

GAE production deploy

cd twitalyse
lein appengine-prepare
cd $GAE_SDK/bin
./appcfg.sh update $twitalyse/war/

Roadmap:

iteration 0 [0/4]

infra [1/3]

Pre-requisites - Development Platform [2/2]

Install leiningen https://github.com/technomancy/leiningen

Install swank-clojure https://github.com/technomancy/swank-clojure

put in place cloudbees [0/3]

creation d’un compte

init job

handle secret files in cloudbees

We don’t want them in the github repo :)

deploy auto dans GAE

technical validation of all components [2/5]

poc clojure / twitter

poc clojure / gae

feedback on the author of appengine-magic: for the sdk 1.4.3

won’t do

deploy on appengine

integration clojure/twitter/gae [1/3]

WARN: blacklisted apis in appengine could pose problems

Yes, https with GAE, doesn’t work (httpsconnection is blacklisted). Even with http it breaks in fact, see this issue.

ask the twitter-clojure guys if the lib is GAE compatible

See issue above.

look for twitter4j (gae ready) [1/3]

local java test

in clojure

in clojure + GAE

poc gae bigtable

Note: we will need to use the GAE datastore (see iteration1) Using this api from clojure may not be trivial …

documentation [/]

write how to deploy on appengine

security: externalize the secret phrases

• If we add them each time in the src files, one day it will by pushed by error :)
• should be in a var env, user home, or something
• Loaded from a file which is in .gitignore

iteration 1 [0/2]

(ugly page) Display the number of twitter tagged #sfeir of a hardcoded list of sfeir accounts. [0/4]

content of the page

• Display as a simple list, with 2 columns :
  • twitter name
  • full person name
  • counter (what about the retweet ? Does it count as a tweet ? KISS! But should be indicated)
  • sorted by counter, then twitter name
• count the tweets since the twitter api gives us (5-7 days according to this thread.
• display a last updated (“last updated 3h ago”)

cache the result of the twitter request

explaination : the twitter query takes a long time (even for one tweet!). So we will need to cache it. It means using the GAE datastore …

look of the page [0/2]

it would be nice to have a pretty html page (design is important)

The sfeirien with the most #sfeir twit highlited (winner!)

powered by logos (powered by clojure/gae/sfeir)

iteration 2 [/]

store the tweets for history since the begining of times