Youtube with https #417

VoidAndAny · 2020-12-16T11:12:04Z

Your lib is used in Bolts CMS (at least 3.x version) to embed Youtube video.
Since ?? weeks/months, it doesn't work anymore with the following error :

[HTTP/1.1 500 Internal Server Error 756ms]
    success	false
    code	500
    error	Object { type: "\\Client", file: "vendor/guzzlehttp/guzzle/src/Exception/RequestException.php", line: 113, … }
    type	"\\Client"
    file	"vendor/guzzlehttp/guzzle/src/Exception/RequestException.php"
    line	113
    message	"Client error: `GET http://www.youtube.com/oembed?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D-7mdBNjaMqc&format=json` resulted in a `403 Forbidden` response:\n{\n \"error\": {\n \"code\": 403,\n \"message\": \"SSL is required to perform this operation.\",\n \"status\": \"PERMISSION_D (truncated...)\n"

Can we access Youtube URL with https :
https://www.youtube.com/oembed

(I made the correction in my vendors and it's OK)

I can make a PR if you want.

Tahnks

The text was updated successfully, but these errors were encountered:

kinglozzer · 2020-12-16T15:38:25Z

I’ve encountered this too, admittedly only over the last couple of days. Is there any reason why some providers use HTTP instead of HTTPS? PR: #418

cleentfaar · 2020-12-16T17:22:17Z

Okay so I've done some digging and found the likely issue.

There are two ways used by Embed to determine the correct embed-endpoint of a given provider.

First it tries to visit the video URL itself and see if there are any <link> tags that indicate the correct URL to use. For example, on a rickroll video you will see this:

<link rel="alternate" href="android-app://com.google.android.youtube/http/www.youtube.com/watch?v=oHg5SJYRHA0">
<link rel="alternate" href="ios-app://544007664/vnd.youtube/www.youtube.com/watch?v=oHg5SJYRHA0">
<link rel="alternate" type="application/json+oembed" href="http://www.youtube.com/oembed?format=json&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0" title="RickRoll&#39;D">
<link rel="alternate" type="text/xml+oembed" href="http://www.youtube.com/oembed?format=xml&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0" title="RickRoll&#39;D">

Note that youtube itself is providing an incorrect protocol in it's uris (http instead of https)! Problem number one found!

Okay, I thought, so YouTube is messing up... But why is Embed even trying these link tags when it has preconfigured providers (Youtube, Vimeo etc) that have been designed to do this as indicated here: https://github.com/oscarotero/Embed/blob/master/src/resources/oembed.php#L932.

The answer to that can be found here: https://github.com/oscarotero/Embed/blob/master/src/OEmbed.php#L56-L60. As you can see there, the detectEndpointFromProviders is only used as a last resort for some reason.

So summarizing:

Youtube is serving a bad url (http protocol) in it's alternate <link> tags on video pages
Embed always tries <link> tags first before using it's built-in providers. If the provider was used it would have ignored youtube's error and use the proper (https) protocol by default.

Now to the actual fix, that's a tougher nut to crack. Here are the options as I see them, with my opinion added:

Let Embed always give priority to it's providers, before trying <link> tags:
- PRO: nice for us devs because it fixes the issues we have now embedding youtube video's, without having to wait for youtube to notice.
- CON: You put even more trust in Embed's pre-configured providers, rather than letting these videosites communicate their own URIs. Not a big problem short-term, but hard to maintain long-term.
Wait for YouTube to fix their <link rel="alternate"> tags:
- PRO: fix problems at the source rather than working around them through this Embed package.
- CON: Waiting for YouTube to change something, whilst having customers complaining that they cant watch embedded videos on our website.

In the meantime my company can't afford to break embedded videos so I will work with a fork until the package author makes a decision on this.

Hopes this helps others that are trying to debug this.

oscarotero · 2020-12-16T20:49:05Z

Ok, I see. Thanks for the valuable info @cleentfaar

I'm going to wait for Youtube. Changing the priority of the detectors may fix this issue but open others. The philosophy of this library is rely first in the data provided by the site (that can change anytime) and only use the pre-configured data as the last resort, because it can become deprecated anytime.
Note also that the embed code is returned even if the oembed endpoint is failing (as you can see in the demo: https://oscarotero.com/embed/demo/index.php?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0&settings=) This is due it's using the video url provided in the og:video:url meta element.

cleentfaar · 2020-12-17T09:12:53Z

@oscarotero not sure I understand what you mean, the demo you link to doesnt work either?

oscarotero · 2020-12-17T09:51:46Z

What I mean is that, even if the oEmbed endpoint fails, we can get a embed code:

because Embed get the video url from opengraph:

cleentfaar · 2020-12-17T10:28:15Z

ok, but thats not what happening in my code at the moment, it just returns null, what should I change to use that opengraph?


        $info = $embed->get($url);
        $oembed = $info->getOEmbed();
        $embedHtml = $oembed->get('html');

are you suggesting doing something with $info->getMeta() instead of $info->getOEmbed(), there's no single method that I can use which does this for me?

oscarotero · 2020-12-17T11:02:12Z

Sorry, I don't understand. Doesn't this code work for you?

$info = $embed->get($url);
$embedHtml = $info->code;

I mean, you don't have to get the different apis directly (oembed, meta, etc). Embed library does it automatically for you, as you can see in the example: https://github.com/oscarotero/Embed#usage

leevigraham · 2021-02-16T08:53:59Z

@oscarotero to add some further clarification:

This works fine:

$info = $embed->get('https://www.youtube.com/watch?v=sRy8aLzZRyA&feature=youtu.be');
$embedHtml = $info->code;

This fails:

$info = $embed->get('https://www.youtube.com/watch?v=sRy8aLzZRyA&feature=youtu.be');
$embedHtml = $info->getOEmbed()->all();

with (json output):

"ombed": {
       "error": {
           "code": 403,
           "message": "SSL is required to perform this operation.",
           "status": "PERMISSION_DENIED"
       }
   }

Following the url directly in the browser: http://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsRy8aLzZRyA performs a 301 redirect. I wonder if the underlying code is not following redirects?

gabrielsolomon · 2021-02-16T10:07:56Z

I tried to fix this in #419

oscarotero · 2021-02-16T10:16:46Z

The function $info->all(); does not exist in this library.

leevigraham · 2021-02-17T03:43:40Z

Sorry that should have been $info->getOEmbed()->all(); updated my comment.

oscarotero · 2021-02-17T10:00:11Z

Ok. This is because this library not only use oEmbed API to get the data, but also html meta tags, linked data, etc.
In this case, due oEmbed endpoint fails, the data is returned by opengraph or twitter cards. You can see the full code detector here: https://github.com/oscarotero/Embed/blob/master/src/Detectors/Code.php

You shouldn't use the oEmbed class directly, but the $info->code property.

This was referenced Dec 16, 2020

fix(youtube): update YouTube endpoint to use https instead of http #418

Merged

Temporarily fixed issue with youtube messing up it's link tags Pararius/Embed#1

Closed

gabrielsolomon mentioned this issue Dec 17, 2020

fix(youtube): add retry for https if http fails in oembed request [#417] #419

Open

cleentfaar mentioned this issue Dec 17, 2020

Temporary fix for youtube embeddables Pararius/Embed#2

Closed

kinglozzer mentioned this issue Dec 21, 2020

Embedding media - videos does not set the correct width / height silverstripe/silverstripe-cms#2625

Closed

bobdenotter mentioned this issue Dec 24, 2020

OEmbed seems wonky. bolt/bolt#7891

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Youtube with https #417

Youtube with https #417

VoidAndAny commented Dec 16, 2020

kinglozzer commented Dec 16, 2020 •

edited

Loading

cleentfaar commented Dec 16, 2020

oscarotero commented Dec 16, 2020

cleentfaar commented Dec 17, 2020 •

edited

Loading

oscarotero commented Dec 17, 2020 •

edited

Loading

cleentfaar commented Dec 17, 2020

oscarotero commented Dec 17, 2020 •

edited

Loading

leevigraham commented Feb 16, 2021 •

edited

Loading

gabrielsolomon commented Feb 16, 2021

oscarotero commented Feb 16, 2021

leevigraham commented Feb 17, 2021

oscarotero commented Feb 17, 2021

Youtube with https #417

Youtube with https #417

Comments

VoidAndAny commented Dec 16, 2020

kinglozzer commented Dec 16, 2020 • edited Loading

cleentfaar commented Dec 16, 2020

oscarotero commented Dec 16, 2020

cleentfaar commented Dec 17, 2020 • edited Loading

oscarotero commented Dec 17, 2020 • edited Loading

cleentfaar commented Dec 17, 2020

oscarotero commented Dec 17, 2020 • edited Loading

leevigraham commented Feb 16, 2021 • edited Loading

gabrielsolomon commented Feb 16, 2021

oscarotero commented Feb 16, 2021

leevigraham commented Feb 17, 2021

oscarotero commented Feb 17, 2021

kinglozzer commented Dec 16, 2020 •

edited

Loading

cleentfaar commented Dec 17, 2020 •

edited

Loading

oscarotero commented Dec 17, 2020 •

edited

Loading

oscarotero commented Dec 17, 2020 •

edited

Loading

leevigraham commented Feb 16, 2021 •

edited

Loading