Automated mass website observation - or a tool to see how fast council websites are.
Pagespeedy is a series of scripts to run the Google PageSpeed Inights against a list of websites - it is used by us to do the Council Page Speedyindex thing
http://jumoo.uk/speedy/sites.php
PageSpeedyPlus comes in two parts
Part One - The Analyser The Python Scripts, do background runs against websites.
You will need.
- A List of sites you want to run it against (see sites.txt)
- To Initialize the DB
- an installation of AChecker to to the Accessibility Checks. (or you can turn them off in the code)
- Some time to run it (it takes about an hour on 400 sites)
Setup:
- Initialize your DB (run
SpeedyPlusInit.py
) - You need 3rd Party APIs
- A Google Page Speed API Key goes in
pagespeed.config.json
(seepagespeedy.config.sample.json
) - If you run AChecker -
achecker.config.json
needs keys too
- A Google Page Speed API Key goes in
- Decide what month your going to do
speedyplus.py -l
will list what months you can run againstspeedyplus.py -m <id>
will run the process against a month- at the end the month is marked as processed and can't be ran again (without some sqlite hackery)
By default pagespeedy will do:
- a Google PageInsights check,
- a Wappalizer check
- an Accessiblity check
To turn off one of these just comment the two lines out in speedyplus.py.
Part 2 is the site
The second part is the website (in website folder) - it's a PHP site.
to run the site you need to copy a few things from your analyser
- Copy the speedyplus.db to the root of your site
- Copy/Move the results folder to the site
The results folder can get quite large that's why we move it (in our case we ftp it up to our site).
- The website is intended to be a read only copy of everything, So set it so if you put it on a web server.
- You will need to poke around in the SQL at some point. just run
Sqlite3 speedyplus.db
in the directory. - This is very rough , things aren't separated properly in the code (it sort of grew) - feel free to contribute improve and feedback.