Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTPError: 503 Server Error: Service Unavailable for url: https://coronavirus-tracker-api.herokuapp.com/v2/locations?country_code=BD&source=jhu #13

Open
mobassir94 opened this issue Apr 7, 2020 · 22 comments
Open

HTTPError: 503 Server Error: Service Unavailable for url: https://coronavirus-tracker-api.herokuapp.com/v2/locations?country_code=BD&source=jhu #13

mobassir94 opened this issue Apr 7, 2020 · 22 comments

Comments

@mobassir94
Copy link

if i try this :
import COVID19Py
covid19 = COVID19Py.COVID19()
loc = covid19.getLocationByCountryCode("BD")

i get error : HTTPError: 503 Server Error: Service Unavailable for url: https://coronavirus-tracker-api.herokuapp.com/v2/locations?country_code=BD&source=jhu
@Kamaropoulos
Copy link
Owner

It turns out this is not an issue with COVID19Py but with the API it pulls data from.

According to ExpDev07/coronavirus-tracker-api's Issue #264, there was an outage and it seems like it's still down at the time I'm writing this.

A temporary workaround would be to use one of the live forks provided by the community on the issue's comments. In order to do that, initialize the COVID19 object as follows:

covid19 = COVID19Py.COVID19(url="<YOUR_ALTERNATIVE_URL_HERE>")

For example, in order to use @toxyl's load-balanced API, you could do the following:

covid19 = COVID19Py.COVID19(url="https://cvtapi.nl")

I hope that helps.

It might also be a good idea to keep a list of mirrors somewhere and have the package use any that is available at the time. This should sort out the issue of the default URL being unavailable and requiring the end-user to provide their own URL every time there is an outage.

@Kilo59
Copy link

Kilo59 commented Apr 8, 2020

@Kamaropoulos @mobassir94 it was only down for about an hour.
Even still it's probably a good idea to have a backup if you need guaranteed availability and/or cache frequently.

@proftc
Copy link

proftc commented Aug 2, 2020

Does anyone know of any other mirrors because the jhu site is often very busy and requests seem to fail quite often

@toxyl
Copy link

toxyl commented Aug 2, 2020

Did you try my mirror at https://cvtapi.nl? there aren't many users on it, so failing requests shouldn't really be an issue.

@fffelix-jan
Copy link

I have this issue too. It also seems to be down for me.

@jdeweese1
Copy link
Contributor

Hi all,

It looks like the data source is now broken.

However, on this issue on the data source repo , they recommended https://covid-tracker-us.herokuapp.com as another data source. I plugged that into the constructor in the latest.py file,

import COVID19Py

covid19 = COVID19Py.COVID19(url='https://covid-tracker-us.herokuapp.com')

print(covid19.getLatest())
print(covid19.getLocationByCountryCode(country_code='US'))
{'confirmed': 37180308, 'deaths': 1072306, 'recovered': 6091449}
[{'id': 242, 'country': 'US', 'country_code': 'US', 'country_population': 327167434, 'province': '', 'last_updated': '2020-10-12T04:54:49.884370Z', 'coordinates': {'latitude': '40.0', 'longitude': '-100.0'}, 'latest': {'confirmed': 7717932, 'deaths': 214370, 'recovered': 0}}]

It seems to pass at least the basic smoke tests. If this is indeed API compatible with our wrapper, then it may not be a bad idea to use it as our new default data source.

@Kamaropoulos
Copy link
Owner

Agreed @jdeweese1.

In the meantime, API sources have proven to be unstable so maybe it might be time to implement source mirrors support?

@jdeweese1
Copy link
Contributor

I went ahead and made a PR using another data source I found #20 .

If we are able to find some more mirrors, that could definitely be a good move for us in the case that another backend fails.

@Kamaropoulos
Copy link
Owner

#20 looks good, just merged it.

https://cvtapi.nl/ also seems to be still available so maybe we could start with these two. In case something happens with the current default mirror, we'll have an alternative that will keep things running without troubling this project's users.

Then, we can look around and find what other mirrors are available, I might set up one too. We could even propose a heartbeat system for the API to have every running mirror report its status every now and then. This way, mirror discovery would be automatic.

@Kamaropoulos
Copy link
Owner

Additionally, something like this could allow us to select a mirror for the user based on their ping to it or even implement load balancing between the mirrors, even though it will probably not make any difference at this scale.

@toxyl
Copy link

toxyl commented Oct 12, 2020

Yes, https://cvtapi.nl/ is still up, but it's not load balanced anymore. I didn't have enough traffic to warrant that. Implementing a load balancing scheme between mirrors sounds like a good idea, it's cheaper for people to spin up a non-loadbalanced instance, a $5 DO droplet does the trick.

@wobsoriano
Copy link

#20 looks good, just merged it.

https://cvtapi.nl/ also seems to be still available so maybe we could start with these two. In case something happens with the current default mirror, we'll have an alternative that will keep things running without troubling this project's users.

Then, we can look around and find what other mirrors are available, I might set up one too. We could even propose a heartbeat system for the API to have every running mirror report its status every now and then. This way, mirror discovery would be automatic.

Instead of using a API, just create a github action that generates a json file based on a cases csv like this and just call that json file directly in your app. You can check my implementation here https://github.com/wobsoriano/covid3d

@Kamaropoulos
Copy link
Owner

Kamaropoulos commented Oct 14, 2020 •
edited
Loading

Yes, https://cvtapi.nl/ is still up, but it's not load balanced anymore. I didn't have enough traffic to warrant that. Implementing a load balancing scheme between mirrors sounds like a good idea, it's cheaper for people to spin up a non-loadbalanced instance, a $5 DO droplet does the trick.

@toxyl Right, this way we could off-load some of the weight from the API providers as we distribute it between them, while also providing the best experience possible to everyone that needs easy access to the data.

@Kamaropoulos
Copy link
Owner

#20 looks good, just merged it.
https://cvtapi.nl/ also seems to be still available so maybe we could start with these two. In case something happens with the current default mirror, we'll have an alternative that will keep things running without troubling this project's users.
Then, we can look around and find what other mirrors are available, I might set up one too. We could even propose a heartbeat system for the API to have every running mirror report its status every now and then. This way, mirror discovery would be automatic.

Instead of using a API, just create a github action that generates a json file based on a cases csv like this and just call that json file directly in your app. You can check my implementation here https://github.com/wobsoriano/covid3d

There are multiple schools of thought on this. Your point that the data can be analyzed, kept, and accessed directly from a Git repository is a valid one and many people did exactly that at the very beginning of the pandemic. But that, for some people might seem a bit too involved and complicated, having to get the data from a repo and do all the filtering/handling of the data themselves. Also, there are people that just need the deltas since the last day, and having to get all the pandemic data for all the countries is kind of a waste if all you need is three integers and a package can already provide that.

Other people will want to download the data manually to do a deeper analysis on them. Others will want to call an API by themselves and get some specific piece of data. Lastly, others will need a simple and out of the box solution that allows them to access that same information without much effort, so they can focus on what matters the most, figure out how the spread of the pandemic evolves over time and what is the real-world effect of our decisions on the fight against COVID-19. And having to implement the same thing over and over again isn't the best use of developers' and data scientists' time.

I believe that all these solutions can co-exist and be there for when people need them.

@Kamaropoulos
Copy link
Owner

#20 looks good, just merged it.

https://cvtapi.nl/ also seems to be still available so maybe we could start with these two. In case something happens with the current default mirror, we'll have an alternative that will keep things running without troubling this project's users.

Then, we can look around and find what other mirrors are available, I might set up one too. We could even propose a heartbeat system for the API to have every running mirror report its status every now and then. This way, mirror discovery would be automatic.

I also set up a mirror a couple of days ago. It's available at https://covid19-api.kamaropoulos.com

@toxyl
Copy link

toxyl commented Oct 14, 2020

Yes, https://cvtapi.nl/ is still up, but it's not load balanced anymore. I didn't have enough traffic to warrant that. Implementing a load balancing scheme between mirrors sounds like a good idea, it's cheaper for people to spin up a non-loadbalanced instance, a $5 DO droplet does the trick.

@toxyl Right, this way we could off-load some of the weight from the API providers as we distribute it between them, while also providing the best experience possible to everyone that needs easy access to the data.

And you can control availability, a simple HEAD request (if allowed by the server) should be enough to figure out if an API is alive and reacting fast enough to be used.

@proftc
Copy link

proftc commented Oct 14, 2020 via email

@Kamaropoulos
Copy link
Owner

https://us02web.zoom.us/j/81393277277?pwd=MUlBMU82V1MyeHJSeC9zZkdGeXBOZz09 Meeting ID: 813 9327 7277 Passcode: 5xHJfR ¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬ Professor Terence Cosgrove TEL +44 (0) 1275 845230 School of Chemistry TEL +1 239 304 5982 Cantock's Close [email protected] Bristol BS8 1TS UK http://www.chm.bris.ac.uk/pt/polymer/pig_home.htm ¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬ From: Tox [email protected] Reply-To: Kamaropoulos/COVID19Py [email protected] Date: Wednesday, October 14, 2020 at 5:53 PM To: Kamaropoulos/COVID19Py [email protected] Cc: Terence Cosgrove [email protected], Comment [email protected] Subject: Re: [Kamaropoulos/COVID19Py] HTTPError: 503 Server Error: Service Unavailable for url: https://coronavirus-tracker-api.herokuapp.com/v2/locations?country_code=BD&source=jhu (#13) Yes, https://cvtapi.nl/ is still up, but it's not load balanced anymore. I didn't have enough traffic to warrant that. Implementing a load balancing scheme between mirrors sounds like a good idea, it's cheaper for people to spin up a non-loadbalanced instance, a $5 DO droplet does the trick. @toxylhttps://github.com/Toxyl Right, this way we could off-load some of the weight from the API providers as we distribute it between them, while also providing the best experience possible to everyone that needs easy access to the data. And you can control availability, a simple HEAD request (if allowed by the server) should be enough to figure out if an API is alive and reacting fast enough to be used. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#13 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACAXD7FKGWLITF6NHZVTI6DSKXJQTANCNFSM4MDIS5LQ.

@proftc How may I help?

@proftc
Copy link

proftc commented Oct 15, 2020 via email

@Kamaropoulos
Copy link
Owner

@toxyl cvtapi.nl seems to be down now, returns 502 for me.

@proftc
Copy link

proftc commented Oct 31, 2020

All of my mirrors seem to be failing. This is a csv file that seems to work OK in python
URL_DATASET = r'https://raw.githubusercontent.com/datasets/covid-19/master/data/countries-aggregated.csv'
df1 = pd.read_csv(URL_DATASET)

@toxyl
Copy link

toxyl commented Nov 1, 2020

@toxyl cvtapi.nl seems to be down now, returns 502 for me.
Thanks for the heads up. The certificate renewal failed. It should be back online a valid certificate.

@vyahello vyahello mentioned this issue Sep 5, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@proftc @Kamaropoulos @toxyl @wobsoriano @Kilo59 @mobassir94 @jdeweese1 @fffelix-jan