Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NOTICE: The Uitzending Gemist tvOS App has reached its End of Life. Use TV Gemist instead. #29

Open
dyve opened this issue Jan 30, 2018 · 20 comments
Open

NOTICE: The Uitzending Gemist tvOS App has reached its End of Life. Use TV Gemist instead. #29

dyve opened this issue Jan 30, 2018 · 20 comments

Comments

@dyve
Copy link

dyve commented Jan 30, 2018

I don't see any new programs appear since last Sunday (Jan 28). The live views work, the existing programs from the past weeks work, but the "By Day" view shows nothing, and the "Program" view shows no new episodes. Did NPO change something?
@MatrixView
Copy link

Yup, the old NPO API (apps-api.uitzendinggemist.nl) is likely being discontinued because the broadcast data requests are producing empty json replies the last few days. Of course it is possible to parse/scrape the npo website, but that beats the purpose of an API... Of which they have a new version, but probably won't give you access to...

@4np
Copy link
Owner

4np commented Jan 31, 2018 •
edited
Loading

Yes, unfortunately the old API -that was replaced by NPO 2.0 in spring 2017- has now finally been pulled offline. While local cache ensures you can still watch some programs you have favourited as well as live TV, this is pretty much the end of where this app can go.

Having said that, I do have a working prototype I built from scratch a while ago with very basic functionality for the new API (browsing programs, filtering them and watching them). Unfortunately the challenge is that it relies on an API key, which the NPO can kill any time. Making it a bit harder is that they use SSL Pinning in the new mobile app, so you cannot reverse engineer the app very well. And, of course, all the other functionality still needs to be written. There are also some other challenges with the API itself, as it cannot do one of the most basic things like sorting alphabetically 😏 I guess they think you need to search if you want to find a particular program instead of paginating through a list? What if you don't want that :)

@martysmartySE
Copy link

I'm actually only seeing SSL Pinning on the latest version of the iPad app, at the point where it needs to retrieve the ApiKey. Once it has that cached, it seems to no longer use that on other requests. I'm able to do a couple requests there. The new API is located at https://start-api.npo.nl. With the parameter ApiKey you can actually supply your API key.

For my internal app I don't need paginating, so I haven't looked into that, but the app should be able to no?

I'll see if I can decode more endpoints during the week.

Do please post if you find out how to retrieve an ApiKey

@4np
Copy link
Owner

4np commented Jan 31, 2018 •
edited
Loading

I have an APIKey, but I am bit hesitant to throw it open in the public considering the NPO might disable it ;)

Pagination / infinite scrolling works much more speedy than through the deprecated API. I have most requests sorted out, there aren't that many. The only part I am having trouble with is getting the new player (using Apple's FairPlay Streaming FPS) up and running. I am currently up to the point that I have the most of the pieces -Fairplay CERT, Server Playback Context (SPC)-, but I am having trouble obtaining the Content Key Context (CKC). If you have any requests you can intercept that successfully set up a FairPlay-stream that would help :) I am getting the impression the NPO apps are using the proprietary player from nepworldwide.nl (that take care of the streams) using DASH / Widevine, so I am not even sure FairPlay Streaming should work...

Relevant API Endpoints:

I am thinking of open sourcing the WIP-app and networking kit (it's all in Swift 4) when I have taken it a bit further. Not sure what to do with the APIKey though... Perhaps I'll include it at some point when I don't need to debug the code anymore.

@MatrixView
Copy link

@4np Wow, that is great stuff. I really hope you'll open this up further...
The apikey probably is consistent across platform/app/user/network, so throwing it "out there" is probably not as bad as you might expect because their own app wouldn't work either if they blocked it. (unless they wrote it so it'll refresh)

Besides... the P in NPO stands for Public. Just sayin'.. :-) They used to have a hackathon a few years ago and many open API's supporting it (NPO Backstage). I kid you not!

@dyve
Copy link
Author

dyve commented Jan 31, 2018 via email •
edited by 4np
Loading

@Reino17
Copy link

Reino17 commented Jan 31, 2018

This is the old api, afaik. The new api looks like https://rs.poms.omroep.nl/v1/api/subtitles/AT_2085754/nl_NL/CAPTION.vtt. Or I've mistaken it for yet another api.

@martysmartySE
Copy link

FYI, it looks like part of the old API is still available. I do a weekly update on all series, starting at http://apps-api.uitzendinggemist.nl/series.json, which seems still available.

Only the daily updates have indeed gone away (https://apps-api.uitzendinggemist.nl/broadcasts/2018-01-30.json)

@MatrixView
Copy link

@MKaterbarg Sure, the old API is still working, but there's less and less data added to it.
The series.json you mention does contain data, but (besides new series) you need details from:
../series/{series-id}.json requests, and those have no episode info (yet) of 30-01-2018 and 31-01-2018.
The latest episodes that were added in the last few days are from 28-01-2018 and 29-01-2018. I know this because I create RSS feeds from this old API. My feeling is that NPO will slowly kill it by not feeding it new data. That way users will seek refuge to new apps and the NPO website.

@martysmartySE
Copy link

martysmartySE commented Feb 1, 2018 •
edited
Loading

Ah thanks @MatrixView . My script hadn't run for a few days, so it still found about 2000 new eps :)

@4np
Copy link
Owner

4np commented Feb 1, 2018

Hah, nice work @dyve, that's a different API Key than I have :) That means I can publish the code I am working on using that publicly available API Key which makes it a lot easier :)

@Reino17, that tt888 endpoint is indeed the old one, the new media API (e.g. https://start-api.npo.nl/media/AT_2083054) provides references to the SRT subtitles.

Like @MatrixView says, the old API is in part still functioning, but the content is lagging behind (if updated at all). When I have some time I'll create a new repo to publish the working code I have right now.

@MatrixView
Copy link

@4np Ah, that'd be great!

NPO seems to have a github channel too... with a different API.
https://github.com/npo-poms

...Unfortunately they didn't slip any APIkey and/or credentials in with that source. :-(

@4np 4np changed the title No new programs appearing NOTICE: The Uitzending Gemist tvOS App has reached its End of Life Feb 2, 2018
@4np
Copy link
Owner

4np commented Feb 2, 2018

I have just published NPOKit. A Swift 4 Framework for interfacing with the NPO 2.0 APIs. Note though that this is still very much alpha and method signatures might change without notice. I have a working tvOS app as well, but I need to clean the code up a bit more before I publish it to GitHub.

@MatrixView
Copy link

@4np Thx for the NPOKit! I will look into it. Although I don't use Swift or even a Mac... But I'm interested in the code and see what I can use for my shell scripts.

BTW. the old API seems indeed lagging, as I've seen "new" episodes of January 30th and 31st roll in. And even some of yesterday. Let's wait and see...

@4np
Copy link
Owner

4np commented Feb 2, 2018

@MatrixView: You can use it from the command line as well. And as Swift if available on Linux that might work as well (I haven't tested that use case yet). Please take a look at the HOWTO for a command line example...

@MatrixView
Copy link

@4np O, thx for the tips but please don't bother! I'm using Powershell and Xidel under Windows. I've taken a quick look at your NPOKit and I'll probably manage to convert your http requests to a script when I have the time to do so.

In the meantime I'm using code from Reino17's BatchGemist to get my TVguide. (This scrapes the NPO-Gids webpage).

Thanx again for your work! If I have any questions I'll know where to find you!
--MV

@4np
Copy link
Owner

4np commented Feb 3, 2018

No worries, I just wanted to make sure that use case was covered as well :) That allows for people to write command line tools as well as by using a Swift server side framework (Vapor, Perfect, Kitura, etcetera) :)

@4np
Copy link
Owner

4np commented Feb 8, 2018

I have just pushed the first working version of TV Gemist. It currently supports paging through programs, filtering programs, paging through episodes and playing episodes. Please take a look :)

@MatrixView
Copy link

@4np Jeroen, ook al heb ik geen AppleTV, bedankt voor het coden van TV Gemist! In dit soort projecten zitten altijd leerzame dingen voor me.

@sanderdg
Copy link

sanderdg commented Feb 13, 2018 •
edited
Loading

I'm just getting started with this wonderful app and I can report that it works great for me.

@4np 4np changed the title NOTICE: The Uitzending Gemist tvOS App has reached its End of Life NOTICE: The Uitzending Gemist tvOS App has reached its End of Life. Use TV Gemist instead. Feb 13, 2018
Repository owner locked as resolved and limited conversation to collaborators Feb 15, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@dyve @4np @sanderdg @MatrixView @Reino17 @martysmartySE