-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
65 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,69 @@ | ||
# infodb | ||
|
||
Infodb aims to solve the problem of feeling out-of-touch with events happening in the world by combining multiple news sources inside of one app. | ||
The app will poll the most popular news sites every so often and group the articles based on the date of publication and associated tags, allowing users to easily get up to date on what's happening (or what has happened some time ago in regard to a specific topic) | ||
## About the project | ||
|
||
As of now the project is in a prototype stage. The final goal is to make it a Spring Boot powered web-app. | ||
infodb aims to solve the problem of feeling out-of-touch with events happening in the world by combining multiple news | ||
sources inside one app. | ||
The app polls the latest news articles from major Polish outlets, such as Onet or Wirtualna Polska, stores them in an | ||
SQL database and provides a REST API allowing the user to easily search through its contents. | ||
|
||
## Capabilities | ||
|
||
Currently, the app supports three news sources: | ||
|
||
- Onet | ||
- Wirtualna Polska | ||
- DonaldPL | ||
|
||
With more planned to be added later. | ||
|
||
It relies on Hibernate for database connectivity, but can be easily expanded to support other database connectors. | ||
By default, PostgreSQL and H2DB drivers are included. | ||
|
||
Articles are fetched from all available sources and saved to all available databases in fixed intervals, which can be | ||
configured by the user. The default delay between fetches is 5 minutes. | ||
The user can also configure how many articles they would like the app to fetch in one go, the default being 20 per | ||
source. | ||
The article ids are cached in a queue to avoid putting too much stress on the databases when checking if the app fetched | ||
an article that is already saved. | ||
|
||
The app exposes a REST API on `[hostname]:[port]/api/v1` with the following endpoints: | ||
|
||
- articles | ||
- A POST endpoint that allows for searching through the database. It requires a request body of the following | ||
format (though no field is necessary and all of them can be null): | ||
```json | ||
{ | ||
"title": "string to be found in the article's title", | ||
"origin": "source of the article, has to be an exact name which can be found on the /api/v1/sources endpoint", | ||
"dateFrom": "an Instant representing the furthest date on which the article could be published", | ||
"dateTo": "an Instant representing the nearest date on which the article could be published", | ||
"tags": ["a", "collection", "of", "tags", "the", "article", "should", "be", "tagged", "with", "(although only one matching tag is enough for a match)"] | ||
} | ||
``` | ||
The endpoint supports pagination with an optional `page` request parameter, with indexing starting from 0. | ||
- articles/latest | ||
- A GET endpoint which returns 20 of latest articles. | ||
The endpoint supports pagination with an optional `page` request parameter, with indexing starting from 0. | ||
- articles/{id} | ||
- A GET endpoint, returning an article matching the provided ID. | ||
- sources | ||
- A GET endpoint, returning information on all the sources infodb uses. Names of the sources required for searching | ||
through the article database can be found here. | ||
|
||
Documentation of the API can be accessed at `[hostname]:[port]/api/v1/api-docs`, with SwaggerUI available on | ||
`[hostname]:[port]/api/v1/swagger-ui.html`. | ||
|
||
## Architecture | ||
|
||
Built using the Java Platform Module System, the app supports loading multiple news sources and database implementations | ||
without having a tight dependency on any of them. | ||
You can add your own news source or database connector by implementing the provided interfaces and adding your module to | ||
the module path on the app's launch - it should be automatically detected by the ServiceLoader! | ||
|
||
This architecture allows for easy expansion of the projects capabilities without modifying the existing code; an | ||
approach I had on the back of my mind throughout the entire process of making the app. | ||
I was heavily inspired by "hexagonal architecture", or "ports-and-adapters", which makes sure that every component of | ||
the app is independent of the other. | ||
|
||
Every article source module and every database module contains unit tests to check if it behaves as expected. |