Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiples configuraciones para queries a twitter #55

Merged
merged 44 commits into from
Mar 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
7853c36
Refactor: parameterizing tweet collecting functions
pepellou Mar 21, 2020
f2d20b8
Command to show the queries in database
pepellou Mar 21, 2020
0b16ee9
Refactor: extract config.js for admin
pepellou Mar 21, 2020
7bcc27a
List of twitter queries for admin
pepellou Mar 21, 2020
ce10541
Admin can add twitter query
pepellou Mar 21, 2020
e0c1eca
Fix script to see queries when optional fields undefined
pepellou Mar 21, 2020
64d6914
Admin can delete twitter query
pepellou Mar 21, 2020
7136c73
Admin can edit twitter query
pepellou Mar 21, 2020
8db9661
Fix strange error
pepellou Mar 21, 2020
d7dc71c
Fix how collect_old value is shown with queries command
pepellou Mar 21, 2020
c8686ef
Collecting tweets from all the multiple queries
pepellou Mar 21, 2020
9893d02
Handle boundaries of multiple queries
pepellou Mar 21, 2020
0a8ccb7
Rename php to backend
pepellou Mar 21, 2020
170ab43
Refactor: extract Database class
pepellou Mar 22, 2020
b65c67c
Refactor: database as an non-static object
pepellou Mar 22, 2020
520bd44
Test Database
pepellou Mar 22, 2020
8ed63ba
Use phpunit from composer in backend
pepellou Mar 22, 2020
192db3a
Database method to get all elements of a reference
pepellou Mar 22, 2020
fcf1f62
Name and description are mandatory for composer validate
pepellou Mar 22, 2020
aa7552d
Prevent test from connecting to real DB
pepellou Mar 22, 2020
ed927d8
Database method to get one element in a collection
pepellou Mar 22, 2020
4535c6d
Move Database.php to specific package
pepellou Mar 22, 2020
8f8a6e1
StringUtils::extractHashtags with blacklist
pepellou Mar 22, 2020
d2caaa3
Message entity
pepellou Mar 22, 2020
d7a4b14
addOne to persist message
pepellou Mar 22, 2020
7706ad1
Refactor script to use db functions
pepellou Mar 22, 2020
2dfbab3
Add namespaces
pepellou Mar 22, 2020
de9caf4
Autoloading all code
pepellou Mar 22, 2020
b4f1068
Twitter service - can get single tweet via API
pepellou Mar 22, 2020
809d01e
Constants for twitter endpoints urls
pepellou Mar 23, 2020
8e06a18
Twitter service - can get list of tweets with filters
pepellou Mar 23, 2020
5f7d778
Refactor: simplify use of variable
pepellou Mar 23, 2020
469eff3
Refactor - StringUtils::extractTags returns array instead of string
pepellou Mar 23, 2020
a9ea8b4
Refactor: output print with colors
pepellou Mar 23, 2020
daccc57
Refactor: extract commands
pepellou Mar 23, 2020
a6a1d0c
Fix boundaries updating when collecting tweets
pepellou Mar 23, 2020
287f7a0
Ensure we collect the most recent tweets
pepellou Mar 23, 2020
d6fd807
Fix bugs in adding tweet from url
pepellou Mar 23, 2020
d7e492b
Improve command output
pepellou Mar 24, 2020
8f351cd
Filter admin messages list by origin
pepellou Mar 24, 2020
4e01444
Origin is 'manual' for manually added messages
pepellou Mar 24, 2020
5696349
Iniciativas.html shows all queries from DB + manual
pepellou Mar 24, 2020
201e4d8
Restructure config for twitter
pepellou Mar 24, 2020
e166f2f
Make test of twitter service independent of config.php file
pepellou Mar 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
php/config.php
php/vendor/
php/composer.lock
php/firebase-credentials.json
backend/config.php
backend/vendor/
backend/composer.lock
backend/firebase-credentials.json
.phpunit.*.cache
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
1) Crea el archivo de configuración copiando de config.php.ini:

```bash
cd php
cd backend
cp config.php.ini config.php
```

Expand All @@ -41,20 +41,20 @@
3) Actualiza las dependencias:

```bash
cd php
cd backend
composer install
```

4) Comprueba que puedes consultar los últimos tweets:

```bash
php php/run.php last
php backend/run.php last
```

5) Configura firebase copiando el archivo firebase-credentials.json.ini:

```bash
cd php
cd backend
cp firebase-credentials.json.ini firebase-credentials.json
```

Expand All @@ -65,7 +65,7 @@
7) Comprueba que puedes recolectar tweets:

```bash
php php/run.php collect
php backend/run.php collect
```

## Instalación local con Docker
Expand All @@ -74,7 +74,7 @@

- docker 19+
- docker-compose 1.25+
- Configurar `php/config.php` y `php/firebase-credentials.json` como se especifica en la sección anterior
- Configurar `backend/config.php` y `backend/firebase-credentials.json` como se especifica en la sección anterior

### Comandos

Expand Down
10 changes: 10 additions & 0 deletions admin/config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
var firebaseConfig = {
apiKey: "AIzaSyCNCfCoMnJMx7ncX6An3zCQc4TWsK60300",
authDomain: "ayuda-alimentos-coronavirus.firebaseapp.com",
databaseURL: "https://ayuda-alimentos-coronavirus.firebaseio.com",
projectId: "ayuda-alimentos-coronavirus",
storageBucket: "ayuda-alimentos-coronavirus.appspot.com",
messagingSenderId: "622677547690",
appId: "1:622677547690:web:0a04757a7c6ab63dacced3"
};

26 changes: 26 additions & 0 deletions admin/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,31 @@
<button data-action="new" >Añadir elemento</button>
</header>

<nav class="navbar navbar-expand-lg navbar-light bg-light mb-4">
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav mr-auto">
<li class="nav-item active">
<a class="nav-link" href="">Mensajes <span class="sr-only">(actual)</span></a>
</li>
<li class="nav-item">
<a class="nav-link" href="queries.html">Búsquedas de Twitter</a>
</li>
</ul>
</div>
</nav>

<h1>Mensajes por procesar</h1>

<div id="filterByOrigin" class="dropdown mb-1">
<button class="btn btn-secondary dropdown-toggle" type="button" id="dropdownMenuButton" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Filtrar por origen
</button>
<div class="dropdown-menu" aria-labelledby="dropdownMenuButton">
<a data-query="all" class="dropdown-item" href="#">Todos</a>
<a data-query="manual" class="dropdown-item" href="#">(introducido manualmente)</a>
</div>
</div>

<table id="messagesToProcess" class="table table-stripped">
<thead class="thead-dark">
<tr>
Expand Down Expand Up @@ -150,7 +174,9 @@ <h5 class="modal-title">Añadir elemento</h5>

<script src="https://www.gstatic.com/firebasejs/7.11.0/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.11.0/firebase-database.js"></script>
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/popper.min.js"></script>
<script src="../js/bootstrap.min.js"></script>
<script src="config.js"></script>
<script src="main.js"></script>
</body>
</html>
Expand Down
42 changes: 21 additions & 21 deletions admin/main.js
Original file line number Diff line number Diff line change
@@ -1,23 +1,9 @@
var init_firebase = function() {
var firebaseConfig = {
apiKey: "AIzaSyCNCfCoMnJMx7ncX6An3zCQc4TWsK60300",
authDomain: "ayuda-alimentos-coronavirus.firebaseapp.com",
databaseURL: "https://ayuda-alimentos-coronavirus.firebaseio.com",
projectId: "ayuda-alimentos-coronavirus",
storageBucket: "ayuda-alimentos-coronavirus.appspot.com",
messagingSenderId: "622677547690",
appId: "1:622677547690:web:0a04757a7c6ab63dacced3"
};

firebase.initializeApp(firebaseConfig);
};

var get_link = function(tweet) {
return tweet.url != undefined ? tweet.url : 'https://twitter.com/' + tweet.nick + '/status/' + tweet.id;
};

$(function() {
init_firebase();
firebase.initializeApp(firebaseConfig);

var database = firebase.database();
var tableOfProcessedMessages = $('table#messages tbody');
Expand All @@ -31,14 +17,15 @@ $(function() {
var tweet = tweets[id];
var link = get_link(tweet);
var tags = tweet.tags != undefined && tweet.tags != '' ? tweet.tags.split(',') : [];
var origin = tweet.origin != undefined ? tweet.origin : '';
var tags_cell = '';
for (var i in tags) {
var tag = tags[i];
tags_cell += '#' + tag + '<br/>';
}
var table = (tweet.gps != undefined) ? tableOfProcessedMessages : tableOfMessagesToProcess;
table.prepend(
'<tr><td>'
'<tr class="filter-by-origin origin-all origin-' + origin + '"><td>'
+ tags_cell
+ '</td><td>'
+ tweet.message
Expand All @@ -47,12 +34,8 @@ $(function() {
+ '" target="_blank">Ver mensaje original</a></td>'
+ '<td><button data-action="edit" data-id="'
+ id
+ '" data-id="'
+ id
+ '">Editar</button> <button data-action="delete" data-id="'
+ id
+ '" data-id="'
+ id
+ '">Borrar</button> '
+ '</td></tr>'
);
Expand Down Expand Up @@ -114,12 +97,29 @@ $(function() {
message: $('#new-modal-message').val(),
tags: $('#new-modal-hashtags').val(),
type: $('#new-modal-volunteer').is(':checked') ? 'volunteer' : 'needHelp',
url: $('#new-modal-url').val()
url: $('#new-modal-url').val(),
origin: 'manual'
};
if (lat != '' && lon != '') {
tweet.gps = { lat: lat, lon: lon };
}
database.ref('tweets').push(tweet);
$('#new-modal').modal('hide');
});

database.ref('queries').on('value', function(snapshot) {
var queries = snapshot.val();
for (var id in queries) {
var query = queries[id];
$('#filterByOrigin > div.dropdown-menu').append(
'<a data-query="' + id + '" class="dropdown-item" href="#">' + query.query + '</a>'
);
}
$('#filterByOrigin > div.dropdown-menu a.dropdown-item').on('click', function() {
$('#filterByOrigin button').html('Origen: ' + $(this).html());
$('.filter-by-origin').hide();
$('.origin-' + $(this).data('query')).show();
});
});

});
114 changes: 114 additions & 0 deletions admin/queries.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
<!DOCTYPE html>
<html lang="es">
<head>
<title>admin - #AyudaAlimentosCoronavirus</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<link href="https://fonts.googleapis.com/css?family=Amatic+SC:400,700|Work+Sans:300,400,700" rel="stylesheet">

<link rel="stylesheet" href="https://code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
<link rel="stylesheet" href="../css/bootstrap.min.css">
<link rel="stylesheet" href="../css/style.css">
<script src="https://code.jquery.com/jquery-1.12.4.js"></script>
<script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
</head>
<body>

<header class="row mb-4">
<button data-action="new" >Añadir búsqueda</button>
</header>

<nav class="navbar navbar-expand-lg navbar-light bg-light">
<div class="collapse navbar-collapse" id="navbarSupportedContent">
<ul class="navbar-nav mr-auto">
<li class="nav-item">
<a class="nav-link" href="index.html">Mensajes</a>
</li>
<li class="nav-item active">
<a class="nav-link" href="">Búsquedas de Twitter <span class="sr-only">(actual)</span></a>
</li>
</ul>
</div>
</nav>

<h1>Búsquedas de Twitter</h1>
<table id="queries" class="table table-stripped">
<thead class="thead-dark">
<tr>
<th>Cadena</th>
<th>Recopilar antiguos?</th>
<th>Recopilados</th>
<th></th>
</tr>
</thead>
<tbody>
</tbody>
</table>

<div id="edit-modal" class="modal" tabindex="-1" role="dialog">
<div class="modal-dialog" role="document">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">Editar búsqueda</h5>
<button type="button" class="close" data-dismiss="modal" aria-label="Cerrar">
<span aria-hidden="true">&times;</span>
</button>
</div>
<div class="modal-body">
<form>
<input type="hidden" id="edit-modal-id" />
<div class="form-group">
<label for="edit-modal-query"></label>
<input type="text" class="form-control" id="edit-modal-query" placeholder="Búsqueda a hacer en Twitter">
</div>
<div class="form-check">
<input type="checkbox" class="form-check-input" id="edit-modal-collect-old">
<label class="form-check-label" for="edit-modal-collect-old">Recopilar antiguos?</label>
</div>
</form>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-primary">Guardar</button>
<button type="button" class="btn btn-secondary" data-dismiss="modal">Cancelar</button>
</div>
</div>
</div>
</div>

<div id="new-modal" class="modal" tabindex="-1" role="dialog">
<div class="modal-dialog" role="document">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title">Añadir búsqueda</h5>
<button type="button" class="close" data-dismiss="modal" aria-label="Cerrar">
<span aria-hidden="true">&times;</span>
</button>
</div>
<div class="modal-body">
<form>
<div class="form-group">
<label for="new-modal-query"></label>
<input type="text" class="form-control" id="new-modal-query" placeholder="Búsqueda a hacer en Twitter">
</div>
<div class="form-check">
<input type="checkbox" class="form-check-input" id="new-modal-collect-old">
<label class="form-check-label" for="new-modal-collect-old">Recopilar antiguos?</label>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-primary">Añadir</button>
<button type="button" class="btn btn-secondary" data-dismiss="modal">Cancelar</button>
</div>
</div>
</div>
</div>


<script src="https://www.gstatic.com/firebasejs/7.11.0/firebase-app.js"></script>
<script src="https://www.gstatic.com/firebasejs/7.11.0/firebase-database.js"></script>
<script src="../js/bootstrap.min.js"></script>
<script src="config.js"></script>
<script src="queries.js"></script>
</body>
</html>

74 changes: 74 additions & 0 deletions admin/queries.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
$(function() {
firebase.initializeApp(firebaseConfig);

var database = firebase.database();
var tableOfQueries = $('table#queries tbody');

var some_count_we_dont_have_yet = 0;

database.ref('queries').on('value', function(snapshot) {
var queries = snapshot.val();
tableOfQueries.html('');
for (id in queries) {
var query = queries[id];
tableOfQueries.prepend(
'<tr><td>'
+ query.query
+ '</td><td>'
+ (query.collect_old ? 'Sí' : 'No')
+ '</td><td>'
+ some_count_we_dont_have_yet
+ '</td>'
+ '<td><button data-action="edit" data-id="'
+ id
+ '">Editar</button> <button data-action="delete" data-id="'
+ id
+ '">Borrar</button> '
+ '</td></tr>'
);
}
$('button').on('click', function() {
var button = $(this);
var action = button.data('action');
if (action == 'edit') {
var id = button.data('id');
database.ref('queries/' + id).on('value', function(snapshot) {
var query = snapshot.val();
if (query != null) {
$('#edit-modal-id').val(id);
$('#edit-modal-query').val(query.query);
$('#edit-modal-collect-old').prop('checked', query.collect_old);
$('#edit-modal').modal();
}
});
} else if (action == 'delete') {
var id = button.data('id');
if (confirm("Seguro de borrar?")) {
database.ref('queries/' + id).remove();
}
} else if (action == 'new') {
$('#new-modal').modal();
}
});
});

$('#new-modal button.btn-primary').on('click', function() {
var query = {
query: $('#new-modal-query').val(),
collect_old: $('#new-modal-collect-old').is(':checked')
};
database.ref('queries').push(query);
$('#new-modal').modal('hide');
});

$('#edit-modal button.btn-primary').on('click', function() {
var id = $('#edit-modal-id').val();
var query = {
query: $('#edit-modal-query').val(),
collect_old: $('#edit-modal-collect-old').is(':checked')
};
database.ref('queries/' + id).set(query);
$('#edit-modal').modal('hide');
});

});
File renamed without changes.
File renamed without changes.
Loading