This microservice was created as a result of the OpenReq project funded by the European Union Horizon 2020 Research and Innovation programme under grant agreement No 732463.
This microservice is responsible for the coordination of all microservices dealing with Twitter data. The main goal of this microservice is to define Twitter accounts that should be continuously observed. In its current state, the microservice first initiates a recurring crawling, classification, and finally storing process of tweets that mention a given account. To achieve this, this miroservice calls the following microservices of OpenReq:
-
Classifier: ri-analytics-classification-twitter
-
Storage ri-storage-twitter
These microservices must be configured and running in order to use ri-orchestration-twitter.
-
Go (→ https://github.com/golang/go)
-
Gorilla Mux (→ https://github.com/gorilla/mux)
-
setup and start the microservices: ri-collection-explicit-feedback-twitter , ri-analytics-classification-twitter, ri-storage-twitter
-
Define a shared base url of all the three previously mentioned microservices called BASE_URL. BASE_URL will be used as an environment variable in the Docker run command (See following section).
-
An SSL certificate must be mounted to the docker image during the run build (see the following example)
-
A bearer token must be added as an environment variable called BEARER_TOKEN
Run the following commands to start the microservice:
-
docker build -t ri-orchestration-twitter .
-
docker run -v "<path_to>/ca_chain.crt:/go/src/app/ca_chain.crt" -e "BASE_URL=<BASE_URL_OF_THE_REQUIRED_MICROSERVICES>" -e "BEARER_TOKEN=<token>" -p 9703:9703 ri-orchestration-twitter
The API is documented by using Swagger2:
This mircoservices orchestrates the communication between several other microservices (crawler, classifier, and storage).
The code currently assumes that all these microservices are behind a reverse proxy.
Therefore, the class rest_handler.go does not specify any ports.
In case you decide to deploy those micoservices without a reverse proxy, we suggest to add the proxies to the endpoints in rest_handler.go.
const (
// analytics layer
endpointPostClassificationTwitter = "/ri-analytics-classification-twitter/lang/"
// collection layer
endpointGetCrawlTweets = "/ri-collection-explicit-feedback-twitter/mention/%s/lang/%s/fast"
endpointGetCrawlAllAvailableTweets = "/ri-collection-explicit-feedback-twitter/mention/%s/lang/%s"
// storage layer
endpointPostObserveTwitterAccount = "/ri-storage-twitter/store/observable/"
endpointGetObservablesTwitterAccounts = "/ri-storage-twitter/observables"
endpointGetUnclassifiedTweets = "/ri-storage-twitter/account_name/%s/lang/%s/unclassified"
endpointPostTweet = "/ri-storage-twitter/store/tweet/"
endpointPostClassifiedTweet = "/ri-storage-twitter/store/classified/tweet/"
)See OpenReq project contribution guidelines